Virtual texture suggestions (to Chris)

[jim]

Hi Chris,

I have some suggestions how Celestia could handle virtual textures.

Before I came to this I have a very big wish. PLease, please, please is it possible that Celestia supports the 8bit palleted DDS format. We need this for high quality normal maps with reduced file size. My test with 64k normal map and u888 DDS format brings Celestia to the limits. I'm not sure if the 8bit format is a problem for the shadder calculation but I hope you can integrate this.

Now to the virtual textures.
I know you want improve the perfomance of this in the next Celestia version. At the moment stops Celestia when it must load textures. Celestia must load all needed textures before it can continue the animation. Now exact this is a problem if you 'sit' on ISS and has some large virtual textures installed (32k surface dxt1, 32k specular dxt1, 64k normal u888, 32k night lights) and set the time speed to 10x, 100x or more. Even the fastet computer gets a strong problem. I guess nobody has 4gb ram installed .

OK this is the situation and now are my suggestions/solution:

Celestia loads only this texture what it can load in realtime. Only when Celestia gets enougth resources and time or the animation is stoped all textures are loaded.

Now in detail:
First there should be a load order for the textures. The surface texture is the most importend and must always loadet. If time and resources remaining than the other virtual textures should be loaded in this order:
clouds
1. nightlights,
2. overlaytexture,
3. specular map,
4. normal map,
If there is a problem with non loaded spec and normal textures use one universal (black) texture.

Only for the surface map should be another rule. We move to the surfave of the planet. Celestia loads at first the level0 tiles. But the next level should be only loaded if there is enougth time and resources free. If this level can be loaded without perfomance problems Celestia tries to load the tiles of the next higher level until the optimal level for the distance is riched. Only at this point should Celestia try to load the other textures.

Another point is that Celestria should try to unload no longer needed texture tiles before the system begins to write this in the swap file.

Ok, Chris this are only some suggestions. But you are the programmer ond only you can decide if something is usefule and possible for Celestia. Maybe you have already better solutions but I hope I could give some inspirations.

I hope you can give an answer to the 8bit palleded format.

Bye Jens

[wcomer]

Chris,

I have also been testing various formats with the high resolution normal map data. Unfortunately the dxt compression was developed for visual data and is not a good format for normal maps; consider that the interpolated colors are often not even valid normal map colors. Dxt1c and dxt3 are almost unusable for normal maps; they leave very obvious artifacts under most lighting conditions, which become greatly amplified during dusk and dawn conditions. In short, a 16k u888 normal map is better than a 32k dxt1c or dxt3 normal map. Currently, I find .png to be the best format for normal maps, because u888 has not only enourmous install size but large disk read times.

I should point out that I'm not convinced that a palletted format will be much of an improvement over the dxt formats. I'm not familiar with the details of this particular .dds format. I assume it is palletted for the entire image. In which case 256 colors may or may not be sufficient for a 512x512 normal map tile. Considering that normal maps have ~4*Pi*128^2 valid colors, the worst case average error for 8-bit palletted is ~10-bits which is worse than the typical case for dxt compression (yes that was an apple to orange comparison.) I have not tried using any 8-bit palletted image format for the high resolution data (Jens, have you tried using 8-bit palletted .png's? If so, how did this compare with u888?)

The basic problem with the dxt compression is that it interpolates 16 bit colors. dxt compression would work much better for normal maps if the texel interpolation used full 24-bit colors. This would only require 2 extra bytes per texel. 10 vs. 8. <sigh>.

Walton