Celestia Forums

At last, I got it right;-).

Since quite some time, I have tried to associate
the hires bump files with highly compressed DXT format, too. The
essential /slowing-down/ of the fps rate in the display of hires bumped
textures came from the fact that up-to-now one had to use *.png or
*.jpg bump files in Celestia 1.2.5.

Here is how the solution works:

In the Celestia code (texmanager.cpp, texture.cpp), the bumpmap files
are converted into so-called /normal maps/ and stored as
such internally, provided a non-vanishing BumpHeight is entered in
solarsys.ssc; i.e. from adjacent pixels, in each point, the normal is
computed that characterizes the bumpyness of the texture. The trick is
now to use NVIDIA's Windows command-line utility, 'NormalMapGen', for
converting /externally/ a hires bump.tga file first into the corresponding
NormalMap.tga file. The latter is a standard 24bit rgb file and in a
second step, is then converted into DXT1c (8:1) format. All one has to
do in solarsys.ssc is to put for Earth

BumpHeight 0.0
BumpTexture "earth-normal.dds"

and analogously for Mars

BumpHeight 0.0
BumpTexture "mars-normal.dds"

whence internally the conversion step to a normalMap is /skipped/ and
earth|mars-normal.dds is correctly interpreted as a normal map;-). Besides
the amazing 8:1 compression, also the lacking conversion from bump
to normal map saves a lot of storage and time, respectively.

The art is now to generate the cleanest and smoothest possible normal
map. For Earth, for example, I start from the original 24bit grayscale
10k SpaceGraphics earth-bump.jpg's and reduce them (cubically) to
8k,4k,2k with GIMP. Without saving the JPG file, I create a new
8k,4k,2k transparent grayscale file and compose a RGBA
earth-bump.tga file with the earth-bump.jpg as alpha channel for each
resolution. Then the crucial command is

NormalMapGen earth-bump.tga earth-normal.tga <scale> -w 5x5

which generates the output earth-normal.tga NormalMap file from the
height info in the alpha channel using 5x5 sampling. The 24bit DDS
conversion call subsequently reads

nvdxt -file earth-normal.tga -24 dxt1c

As result, earth-normal.dds is written. With the scale factor <scale>
one has to play a little. I found <scale> = 7.5, 4.5 satisfactory for 8k, 4k
normal maps of Earth, for example.

That's it! As an illustration, I display below 2 hires Earth shots and one
from Mars. The 1st earth shot has a /16k earth.dds/ texture and a /8k
earth-normal.dds/ texture, the second earth shot has a /8k earth.dds/
texture and the same /8k earth-normal.dds/. The Mars shot was a 8k
mars.dds texture with a 8k mars-normal.dds texture.

Most importantly, watch the /fps-rates/ in the lower left of the
shots!!! This is just amazing, given my old 32MB GF2 card...Moreover,
as you may e.g. judge from the second earth image, the display of the
elliptical plane ground is completely free of pixel trash that is so
often visible there upon normal bumpmapping!

As soon as I find time, I shall deposit the new normal maps on my
TextureFoundry site for download. Of course, Grant's corrections are
incorporated for Mars.

Bye Fridger

OUAAAAOU....

The result is very great

Fridger...your a genius...

Guess I have some playing around to do myself...

It's breathtaking. Especially the two last... I need a moment to take back my mind.

Fantastic! That is some clever work!

In case someone is interested:
The first image below is a 8k grayscale bumpmap of the Himalaya region close to the second earth image above. The second image is what the NormalMapGen tool makes of it: a normal map!

Bye Fridger

Hi Fridger,

i'm happy that you find a way to use dds-files for bumpmaping. But your way is a little bit difficult - it goes easier.
There is only one command to create the needed normal map:

nvdxt -file earthbump2k.tga -n5x5 -rgb -scale 4.5 -24 dxt1c

The result is a earthbump2k.dds.


Ok now for all in detail:

We need "NVDXT.EXE" from NVIDIA's command-line-tools. Copy this in a directory with the bumpmap-file. This is the grey-scale-pictuer we are already use in Celestia for bumpmaping but we have to convert this to TGA-Format (Use an image-editor or -viewer to save in TGA-format). Now we can go to commandline and put in our command or we create a bat-file with our command. Ok we made a bat-file. Open a new file in Nodepad, write

nvdxt -file earthbump2k.tga -n5x5 -rgb -scale 4.5 -24 dxt1c

in it and save as makeddsbump.bat in our directory (were NVDTX.EXE and the bumpmap is). The advantage of a bat-file is we can easy adapt filename and bumphigh (scale 4.5) with Notepad. A double-click on this bat-file create the dds-File in our directory which can be now used in Celestia.

Copy the dds-file in the medres- or hires-directory and edit solarsys.sys. It's absolut necessary that you set BumpHeight 0.0 if you use a dds-bumpfile or Celestia could crash.

Bye Jens

In case someone is interested:

The first image below is a 8k grayscale bumpmap of the Himalaya region close to the second earth image above. The second image is what the NormalMapGen tool makes of it: a normal map!

Bye Fridger

PS: Above I misstyped the second image and could not edit it anymore, since my identity was gone...

jim wrote:Hi Fridger,

i'm happy that you find a way to use dds-files for bumpmaping. But your way is a little bit difficult - it goes easier.
There is only one command to create the needed normal map:

nvdxt -file earthbump2k.tga -n5x5 -rgb -scale 4.5 -24 dxt1c

The result is a earthbump2k.dds.


Ok now for all in detail:

We need "NVDXT.EXE" from NVIDIA's command-line-tools. Copy this in a directory with the bumpmap-file. This is the grey-scale-pictuer we are already use in Celestia for bumpmaping but we have to convert this to TGA-Format (Use an image-editor or -viewer to save in TGA-format). Now we can go to commandline and put in our command or we create a bat-file with our command. Ok we made a bat-file. Open a new file in Nodepad, write

nvdxt -file earthbump2k.tga -n5x5 -rgb -scale 4.5 -24 dxt1c

in it and save as makeddsbump.bat in our directory (were NVDTX.EXE and the bumpmap is). The advantage of a bat-file is we can easy adapt filename and bumphigh (scale 4.5) with Notepad. A double-click on this bat-file create the dds-File in our directory which can be now used in Celestia.

Copy the dds-file in the medres- or hires-directory and edit solarsys.sys. It's absolut necessary that you set BumpHeight 0.0 if you use a dds-bumpfile or Celestia could crash.

Bye Jens

Jens:

Of course, I am aware of the normal map option in nvdxt. I just forgot
to mention that, too. The crucial point was anyway to set BumpHeight 0.0.

But since I usually have high standards as to image quality, the
separate generation and /inspection/ of the normal
map before dds conversion was crucial in my investigations. I studied a number of /inequivalent/ sequences in producing the normal map from a 10k earth-bump.jpg from SpaceGraphics. With your lores 2k example you will barely note any differences;-).

Examples: You may argue that the best normals are generated from
taking the highest resolution available in the bump map. This would lead you to produce a 10k normal map first and reduce the final normal map to 8k,4k,2k. The other (and better) option is to first reduce the bump.jpg to 8k,4k,2k and do 8k,4k,2k normal maps thereof. Big differences for the hires options!

You may also see that there are differences whether you enter the bump map via the alpha channel or via average rgb (as you did it in your 2k example)

and so on...

Once one knows how things work, it is clearly easy and useful to simplify the procedure into *.bat scripts...


Bye Fridger

Fridger, where are the shots of the second picture from?

jim wrote:
Copy the dds-file in the medres- or hires-directory and edit solarsys.sys. It's absolut necessary that you set BumpHeight 0.0 if you use a dds-bumpfile or Celestia could crash.

Bye Jens

If this is the case can this be fixed in a future version of Celestia? I like to have more control over this...

Anyways I will give this a shot too...Thanks Fridger, Jim...

Fridger,

You wrote

Fridger wrote:This would lead you to produce a 10k normal map first and reduce the final normal map to 8k,4k,2k. The other (and better) option is to first reduce the bump.jpg to 8k,4k,2k and do 8k,4k,2k normal maps thereof. Big differences for the hires options!

I haven't tried this yet, but I'm surprised. Since the normal map has 24 bits of information, I would have expected that reducing the normal map would produce better results than reducing the 8 bit greyscale bumpmap. Was the same (or similar) algorithm used in both cases?

selden wrote:Fridger,

You wrote

Fridger wrote:This would lead you to produce a 10k normal map first and reduce the final normal map to 8k,4k,2k. The other (and better) option is to first reduce the bump.jpg to 8k,4k,2k and do 8k,4k,2k normal maps thereof. Big differences for the hires options!

I haven't tried this yet, but I'm surprised. Since the normal map has 24 bits of information, I would have expected that reducing the normal map would produce better results than reducing the 8 bit greyscale bumpmap. Was the same (or similar) algorithm used in both cases?

Fridger's approach is definitely the correct one . . . Remember that in a normal map all of the texels should be normal i.e. sqrt(r*r + g*g + b*b) = 1. The scaling filter in image manipulation programs is general, and does not preserve this property of the texels . . .

--Chris

I just used the nvidia tool alone with spectacular results...Looks just as good as the original...

But im sure there might be benefits from using the long way in some cases...

selden wrote:Fridger,

You wrote

Fridger wrote:This would lead you to produce a 10k normal map first and reduce the final normal map to 8k,4k,2k. The other (and better) option is to first reduce the bump.jpg to 8k,4k,2k and do 8k,4k,2k normal maps thereof. Big differences for the hires options!

I haven't tried this yet, but I'm surprised. Since the normal map has 24 bits of information, I would have expected that reducing the normal map would produce better results than reducing the 8 bit greyscale bumpmap. Was the same (or similar) algorithm used in both cases?

Apart from the relevant issue of loosing the normalization of the normal map after rescaling from 10k size e.g. to 8k, you may have misunderstood:

The color depth is /never/ changed, it remains always at 24bit. I was taking about /rescaling/ the file size from 10k to 8k (not 8bit).
The starting bump.jpg from SpaceGraphics is a 24bit grayscale file (in reality, I think the original height info is 16 bit numbers).

Bye Fridger

chris wrote:Fridger's approach is definitely the correct one . . . Remember that in a normal map all of the texels should be normal i.e. sqrt(r*r + g*g + b*b) = 1. The scaling filter in image manipulation programs is general, and does not preserve this property of the texels . . .

--Chris

Exactly!


It reminds me though that nvdxt has an option (-norm) to normalize normal maps! So, tonight, I might get back to generating a 10k normal map first from the 10k bump.jpg, rescale it to 8k,4k,2k and finally normalize the resulting normal maps with nvdxt...

Bye Fridger

Rassilon wrote:If this is the case can this be fixed in a future version of Celestia? I like to have more control over this...

Anyways I will give this a shot too...Thanks Fridger, Jim...

Ras:

Either me or Chris will incorporate very soon into the CVS tree a new "NormalMap" keyword that is to be used in solarsys.ssc in case of (dds) normal maps. This is definitely more memnonic than remembering BumpHeight 0.0 and assigning a /normal map/ dds texture to the (incorrect) Keyword: "BumpMap";-)

Bye Fridger

Above this was me, again loosing identity...

Ras:

In addition the crashing mentioned by JIM is not due to a bug in the code, but due to improper operation.

In the code there is a branching:

if (bumpheight == 0 )
{

"skip internal conversion to normal map"

} else {

"do internal conversion to normal map"
}

Obviously, if your input texture is not a bumpmap but a normal map and you forget to assign bumpheight=0,
the code tries to convert your normal map again into a normal map ==> crash.

Bye Fridger

Yeah that explains it...Thanks Fridger...

Hi Fridger,

Why is your grayscale bumpmap invers ? My grayscale bumpmaps have white valey and black hills.

t00fri wrote:With your lores 2k example you will barely note any differences;-).

Yes but it was only an example and if i try something i use first lowres textures.

t00fri wrote:You may also see that there are differences whether you enter the bump map via the alpha channel or via average rgb (as you did it in your 2k example)

I didn't use a color-(earth)-map i used the grayscale bumpmap (nvdxt: grey_scale_height_field). I can also use this switches
-red -green -blue the result will be the same. Or i could use -alpha if the bumpinformation is in the alpha channel.


Now a question: work your normal-map correct i mean have shadows and light the real orientiation ? I have some truble with Celestia. Bumpmaping works only fine if i switch vertexshader off. But nightlight is not fine. Vertexshader works only if i switch bumpmaping on. If bumpmaping and vertexshader is on all because bumpmaping work fine even haze. I thought haze can't used on a geforce2. I don't now if this is a problem of my hardware (duron,256mb,geforce2), system (win98se), driver (4072) or celestia (1.2.5).

Now some good news. I played a little bit with the generated normal-maps and split the RGB-channels. The complete information is stored in red- and green-channel where red has the horizontal- and green the vertical-values. If you lock close you will see that the picture of one channel is that what photoshop made with the relief-filter from a grayscale bumpmap. The trick is that the red-channel needs an angel of 0° and the green-channel 90°. Further there must be a temporary border added to the picture which have to be cut after filtering or you will get a line on the mapborder. This happens also if someone use a sharpness-filter on a planet-map (look at the yellow line on jupiter).

Why i tell you about this way ? The quality of such a normal-map is much better than this one made with Nvidias-commandline-tools :-)
Another thing, the blue-channel must be fill with white-color (grayscale) because black mask the planet-map. I believ this can be used for specular-infomation this makes sence but i don't know. Please check the Celestia code.

Sorry i can't present pictures i have no (linkable) webspace not yet.

I have built a test-normal-map (dds-format, 1K) with rings and boxes to check if bumpmappings works correct. If there is an interest i will make this available.

Bye Jens