Shrinking VT resolution in polar latitudes

Tips for creating and manipulating planet textures for Celestia.
Avatar
Topic author
t00fri
Developer
Posts: 8772
Joined: 29.03.2002
Age: 22
With us: 22 years 7 months
Location: Hamburg, Germany

Shrinking VT resolution in polar latitudes

Post #1by t00fri » 08.03.2006, 22:40

Hi all,

this is a first attempt to reduce the reported rendering problems of Fightspit's MONSTER VT tile set of the 128K Earth based on Blue Marble 2nd Generation.

Here is Chris' expert evaluation:

Chris wrote:
This seems to be due to Celestia not managing the texture memory well
enough. The 2k x 2k PNG tiles are very large, each requiring 8MB (2k * 2k
* 4 bytes per pixel) of texture memory. Near the poles, more texture tiles
will be simultaneously visible--just enable wireframe mode and observe the
high density of triangles near planet poles to get an idea of why this is
so. If you're very near a pole and close enough to the Earth so that level
4 tiles are used, at least 32 8MB tiles will need to be made resident
simultaneously. That's as much total video memory as there is on the
card--you just can't have that many textures in use.

One (partial) workaround is what I have just completed

Chris wrote:
Reduce the resolution of tiles nearer the poles. The horizontal
resolution of the tiles near the poles can be reduced. The circumference
at a latitude is cos(latitude) * equatorial circumference. At level 5, a
planet is sectioned into 32 latitude bands, each of 5.625 degrees; thus
the horizontal resolution of the northern and southernmost tiles could
reasonably reduced to cos(90 - 5.625) * equatorial tile resolution. That's
~200 pixels for 2k tiles (which should be rounded up to the nearest power
of two, 256.) Less dramatic savings could be achieved elsewhere. Note that
the vertical resolution of the tiles should be constant.


OK here we go:

I wrote a neat zsh-script that automatically reduces the (horizontal) tile sizes in polar latitudes (see below).
The script should also work under CYGWIN/Windows but needs the zsh (because of its math-module).
The script also needs the 'convert' command line application of ImageMagick that exists for all OS's.

I have optimized the horizontal sizes of Fightspit's 128k Earth tiles (2k x 2k) with this script. The savings are substantial:

++++++++++++++++++
4.8GB ==> 3.7 GB !!
++++++++++++++++++

While the previously reported rendering errors (black tiles) in levels 4,5 have become much more rare, they still occur as well as occasional crashes. The latter clearly indicate a bug in the code, since crashes should NEVER occur ;-) . There is NO visible reduction in image quality!

Calling the script without parameters (or with --help) produces this output:

--------------------------------------------------------------------
Usage: texopt [--help | <level> <tile size> <tile format>]


The shell script 'texopt' is a tool for Celestia > 1.3.0
that optimizes virtual texture tile resolution as function of latitude.

The script requires the standard square tiles tx_i_j
with size <tile size> [px] equal to a power of two,
in a specified format, <tile format> = png, tga, jpg,....

Besides Linux/Unix, the script also runs in a current Cygwin installation
under Windows, ( http://www.cygwin.com ). The zsh is needed!

The script assumes that a recent version (>= 6.1. 8 ) of the ImageMagick package
( http://www.ImageMagick.org ) is installed (either under Unix/Linux or Windows).
The utility 'convert' of that package is used.

Author: Dr. Fridger Schrempp, fridger.schrempp@desy.de
Version: 1.00, 06/03/06
---------------------------------------------------------------------

After typing e.g. in the level 4 directory:

Code: Select all

> texopt 4 2048 png

the typical output is

----------------------------------------------------

Code: Select all

Level =  4
Tilesize =  2048
Number of tiles = 512
Image format of tiles: png

Latitude   Exact multiple   2^n rounding
[degs]      of 128 pix
----------------------------------------
+-84.375     1.5682738989        2
+-73.125     4.6445545474        4
+-61.875     7.5423475641        8
+-50.625     10.1502923852        16
+-39.375     12.3681671508        16
+-28.125     14.1107401749        16
+-16.875     15.3110453515        16
+-5.6249     15.9229556245        16

In case the exact reduced tile size is less than 20% above the next /lower/ power of two, the latter is
given preference over upward rounding (see 2nd entry in the table above). This amounts to a slight effective loss of resolution for the respective row of tiles, but to a much higher saving of space!

Oh I forgot:
=======
The script creates a new subdirectory 'opt' in the levelx directory and outputs a complete new set of optimized tiles in there. This happens either by an appropriate horizontal size reduction or just by copying in case no size reduction is due.


Bye Fridger

Here is the script for people to play....

Code: Select all

-----------------------------texopt-------------------------------------------

#! /usr/bin/zsh
if [ $# -lt 3 -o "$1" = "--help" ]; then
   echo
     echo 'Usage: texopt [--help | <level> <tile size> <tile format>]'
     echo
     echo
     echo The shell script \'texopt\' is a tool for Celestia \> 1.3.0
     echo that optimizes virtual texture tile resolution as function of latitude.
     echo
     echo The script requires the standard square tiles tx_i_j
    echo with size \<tile size\> \[px\] equal to a power of two, 
     echo in a specified format, \<tile format\> = png, tga, jpg,....
     echo
     echo Besides Linux/Unix, the script also runs in a current Cygwin \
       installation
     echo under Windows, \( http://www.cygwin.com \). The zsh is needed!
     echo
     echo The script assumes that a recent version \(\>= 6.1.8\) of the ImageMagick package
     echo \( http://www.ImageMagick.org \) is installed \
        \(either under Unix/Linux or Windows\).
     echo The utility \'convert\' of that package is used.
     echo
     echo Author: Dr. Fridger Schrempp, fridger.schrempp@desy.de
     echo Version: 1.00, 06/03/06
     echo 
else
   zmodload zsh/mathfunc
   level=$1
   tilesize=$2
   tileformat="$3"
   (( p2 = 2**level ))
   echo
   echo "Level = " $level
   echo "Tilesize = " $tilesize
   echo "Number of tiles =" $(( 2 * p2 * p2 ))
   echo "Image format of tiles:" $tileformat
   echo
    echo "Latitude " " Exact multiple  "  "2^n rounding"
    echo " [degs]  " "   of 128 pix    "
    echo "----------------------------------------"   

   mkdir opt
   cd opt

   j=0
   pi=3.1415927

   (( latband = pi/p2 ))
   while (( j  <=  p2/2 - 1 )); do
       i=0
       # calculate the latitude reduction in the /middle/ of the tile
       (( redsize =  cos(0.5 * pi - (j + 0.5) * latband) * tilesize/128.0 ))
       #
       # Returns the next power of 2 if redsize isn't 2^n
       # or returns redsize if redsize is 2^n
        #
        power2=1
       while (( power2 < redsize )); do
          (( power2 <<= 1 ))
       done
       #
       # if redsize exceeds the smaller power of two by less than 20%
       # the latter is used.
       #
       if [ "$(( (redsize - power2 / 2) / redsize ))" -lt 0.2 ]; then
          (( power2 = power2 / 2 ))
       fi
       
       echo "+-"$(( 180*( 0.5  - (j + 0.5) * latband/pi ) )) "    "$redsize "      " $power2
        
       (( redsize = 128 * power2 ))
       (( jm = p2 - 1 - j ))
       if [ "$redsize" -lt "$tilesize" ]; then
          while (( i  <= 2 * p2 - 1 )); do
             convert ../tx_"$i"_"$j".$tileformat -resize "$redsize"x"$tilesize!"  tx_"$i"_"$j".$tileformat
              convert ../tx_"$i"_"$jm".$tileformat -resize "$redsize"x"$tilesize!"  tx_"$i"_"$jm".$tileformat
              (( i++ ))
          done
       else
          while (( i <= 2 * p2 - 1 )); do
             cp ../tx_"$i"_"$j".$tileformat tx_"$i"_"$j".$tileformat
             cp ../tx_"$i"_"$jm".$tileformat tx_"$i"_"$jm".$tileformat
              (( i++ ))
          done
       fi
       (( j++ ))
    done
fi

Last edited by t00fri on 09.03.2006, 08:22, edited 2 times in total.

Avatar
Topic author
t00fri
Developer
Posts: 8772
Joined: 29.03.2002
Age: 22
With us: 22 years 7 months
Location: Hamburg, Germany

Post #2by t00fri » 08.03.2006, 23:03

Hi again,

here is a display from an image viewer, from which you can see the strongly varying horizontal sizes of the various .png tiles of Fightspit's 128k Earth after my script has been applied. Note the thumbnails ON THE RIGHT!

Note that there is NO quality reduction but a reduction of the total size on the hd by > 1 GB...

Bye Fridger

Image

Fightspit
Posts: 510
Joined: 15.05.2005
With us: 19 years 6 months

Post #3by Fightspit » 09.03.2006, 08:19

Hi T00fri,

Your script can be used for people who haven't got a very fast computer and I think it is good solution to improve the performance in Celestia.

Will you want tu upload your script in the Motherlode ?
Motherboard: Intel D975XBX2
Processor: Intel Core2 E6700 @ 3Ghz
Ram: Corsair 2 x 1GB DDR2 PC6400
Video Card: Nvidia GeForce 8800 GTX 768MB GDDR3 384 bits PCI-Express 16x
HDD: Western Digital Raptor 150GB 10000 rpm
OS: Windows Vista Business 32 bits

steffens
Posts: 162
Joined: 06.11.2003
With us: 21 years
Location: RP Germany

Post #4by steffens » 09.03.2006, 08:25

Hi toofri,

reducing the resolution of the image tiles in polar regions was something I always wanted to try out. Now I can do it in a nicely automated way!
Many thanks for your script, I will try it as soon as possible.

steffens

Avatar
Jeam Tag M
Posts: 540
Joined: 01.04.2003
Age: 60
With us: 21 years 7 months
Location: Southern suburb, Paris, France

Post #5by Jeam Tag » 09.03.2006, 09:10

Fightspit wrote:Hi T00fri, Your script can be used for people who haven't got a very fast computer and I think it is good solution to improve the performance in Celestia.
Hello all, this seems to my little computer and I a very, very good idea, I'll try it. Thanks to Chris to point the polar problem and Fridger for his great resolution. Fightspit, could you have a look on the French forum http://www.ikiru.ch/celestia/forum/index.php?act=SF&f=5 where your magnificent textures were also commented... Some users were afread of the big size of your addons, so if you could explain in French this little manip, this would be very helpful :lol: Encore bravo, Jeam
Catalogue des ajouts /Catalog for the Add-Ons in French
...PAGES LOSTS, SORRY

Avatar
Topic author
t00fri
Developer
Posts: 8772
Joined: 29.03.2002
Age: 22
With us: 22 years 7 months
Location: Hamburg, Germany

Post #6by t00fri » 09.03.2006, 09:26

Fightspit wrote:Hi T00fri,

Your script can be used for people who haven't got a very fast computer and I think it is good solution to improve the performance in Celestia.

Will you want tu upload your script in the Motherlode ?


No, I have stopped collaborating with the Motherlode people quite a
while ago due to a severe "personal incident". None of my work is to
be found on ML. After some further testing, I shall upload the script
to my usual place: the TexFoundry.

I will next convert the new size-dependent tiles to DXT1 with my
superfast command-line utility (texconvert) based on the DeVIL
graphical library. Unfortunately, I suppose Chris' machine on which my
TeXFoundry is located, will not be suitable to handle the expected
load if I'd put the DXT1 tiles there...

Bye Fridger


Return to “Textures”