An FAQ on DDS files for Windows users

[The Singing Badger]

Hello Celestians,

I have written the following for Windows users like myself who know nothing about image manipulation but are trying to learn how to use .DDS textures in Celestia.

For many of you, what I have written will seem like the bleeping obvious. But it wasn't bleeping obvious to me. There is plenty of information out there, but it's hard to find and is not available in one single place. I therefore decided to write this FAQ in simple language to save others the fuss and bother that I went through.

For the record, I am using Windows XP with a Radeon 7000 graphics card.

Please note: I'm not an expert, and if anyone reading this FAQ sees anything wrong or foolish, or knows some better alternatives to my suggestions, please say so, and I'll edit them into the post.

Introduction

I intend the following to be a useful FAQ for the computer illiterate. However, I'm assuming that readers have a basic knowledge of how to make changes to Celestia - where to put texture files, how to edit an .ssc file, etc. If you don't know, there is plenty of documentation available at the Celestia Motherlode - http://www.celestiamotherlode.net/catal ... ation.html


What is a .DDS file?
DDS files are a special type of image file used by some graphics cards.

Advantages
They can give you more detailed textures and Celestia can load them faster than JPG images, making the program run smoother.

For example, my graphics card can only handle .JPG files up to a maximum size of 2048x1024 pixels. Anything bigger than that causes Celestia to crash or move impossibly slowly. But with .DDS textures I can go up to 8192x4096 pixels before this happens.

Disadvantages
Converting from JPG to DDS format results in a loss of image quality; the difference manifests primarily in the depth of colour. The differences are small and may not be noticed by the casual user, but if you take your textures seriously this may be a problem.

Although .DDS textures load quickly when being used by Celestia, they take up more disk space than other file formats and load slowly in programs that don't use your graphics card; so, when looking at them with image viewing software, you can expect some long pauses as your computer loads the image.

Will .DDS files work on my graphics card?

They were designed for NVidia cards, but some ATI cards can use them as well.

Celestia can tell you whether or not your card supports .DDS textures. Go to the 'Help' menu and select 'OpenGL Info'. You will be presented with list of OpenGL extensions that your card is fitted with; if "GL_EXT_texture_compression_s3tc" is present, your card supports .DDS.

How can I view .DDS files in Windows?

The major problem is that Windows doesn't know what .DDS files are. This means that none of the standard Microsoft graphics programs can open them, and they won't show as thumbnails in your folders.

However, two freeware programs are available to deal with this problem.


First, you should definitely download IrfanView. - http://www.irfanview.com/ - This is a simple program that enables you to view and resize graphics files, including .DDS files. Using this program you can look at the files without having to start up Celestia - which is handy. Warning: although DDS files load very swiftly in Celestia, they take a long time to load into a graphics viewer - it will take several minutes for a 16k DDS file to load into IrfanView. Don't look at them unless you really want to.

Second, you might want to go to this NVidia website - http://developer.nvidia.com/object/nv_t ... tools.html - and download the DDS Thumbnail Viewer. This is a program that will enable .DDS textures to show up as thumbnails when you're looking at folders in Windows Explorer. Another warning: it takes the computer ages to produce these thumbnails, so long that you may well get annoyed and switch the folder back to normal icon view. It's frustratingly slow, but I guess this may still be a useful utility at times; you decide.

How do I install .DDS textures in Celestia?

This should present no difficulties to anyone experienced in Celestia, but if you're new to it, the following should help:

If you simply want the .DDS texture to be the default one, the simplest method is to place the texture into the 'highres' folder, making sure to rename it by the name of the planet or moon in question (e.g. 'earth.dds', 'europa.dds' etc. Then, once Celestia is launched, type a capital 'R', to select the textures in the 'highres' folder.

If you want to be able to switch between different textures, the simplest method is to install the .DDS as an Alternate Surface. Using a text editor, simply create a new .ssc file containing code like the following and place it in your 'extras' folder.

Code: Select all

AltSurface "Earth DDS texture" "Sol/Earth"
{
   Texture "earth_dds_texture.dds"
}

This will enable you to compare your textures by right-clicking on the planet in question and selecting from a list of Alternate Surfaces.

How can I convert .JPG files to .DDS?

You may want to convert some of your .JPG files into .DDS to speed up Celestia. To do this you need special software, and you need to understand the exact kind of compression necessary.

The Software

There are various freeware programs that can convert files to .DDS. I have described them in order of user-friendliness.

Adobe Photoshop has a plugin that enables you to save files in DDS format - download it from here - http://developer.nvidia.com/object/nv_t ... tools.html

For non-Photoshop users, The ATI Compressonator is the best option for the computer illiterate. It is easy to use (although the resulting image quality may be less impressive than other available software). Go to this website - - http://www.ati.com/developer/compressonator.html - and download ATI's 'Compressonator' [sic]. This program does many things but all that matters for our purposes is that it converts JPG and PNG files to DDS.
The method is simple. Using Compressonator, open a JPG or PNG file. From the 'Compress' menu, select 'DirectX Texture Compression'. You must then choose your compression method from a list (see below for details; normally, you'll want 'DXT1'. The compression process take a while to complete but once it's done you'll get a split screen with the JPG file on the left and a new DDS on the right. Now go to 'File' and select 'Save Compressed'. This will save the DDS version of your original JPG.

DevIL is an Open Source graphics program downloadable from this website - http://openil.sourceforge.net/ Again, it is considered fast the resulting image quality is not perfect. This program is considered the best for handling the largest textures.

NVidia's compression software is considered to produce the best quality DDS images. However, it runs only from a command line (there is no Windows interface), so it might scare off those inexperienced in computing. Also, there is a size limit on files that can be handled. To download it, go to this website - http://developer.nvidia.com/object/nv_t ... tools.html - and download ' DDS Utilities'

Choosing the Correct Compression Method

The compression utilities all require you to choose a compression type. There are three basic types: DXT1, DXT3, and DXT5 (sometimes others in between are listed too). The difference is in whether an 'Alpha Channel' (a separate grayscale layer that can be used to define such things as transparency and specularity) is saved with the file.

If you are converting an ordinary planetary surface texture, you don't need an alpha channel, so choose DXT1. This will save the file without an alpha channel, thus saving space.

The alternative methods will save an alpha channel. DXT3 saves the file with an uncompressed lower quality alpha channel; DXT5 saves it with a compressed but higher quality alpha channel (the file size is the same for both formats). You will only need to use these formats if you are (a) saving a cloud map (because cloud maps use the alpha channel to define the transparent sections of the map), (b) saving a texture that belongs to a 3d model (for the same reason), or (c) saving a texture with specularity defined in the alpha channel.

Normalmaps: Experienced Celestia users reccomend that for Normalmaps you use DXT5 because the high quality of that format is necessary (but PNG is considered best for these textures). See the more detailed postings below this FAQ.

What is a mip-map?

Mip-maps are a series of smaller versions of the texture; they are stored within your DDS file and have the effect of smoothing the transitions as you move toward a texture. Experienced Celestia users reccomend that you always create mipmaps when generating a DDS file (see the more detailed postings below this FAQ). Most compression software includes a function that enables you to create mipmaps (for example, the ATI Compressonator has a dialogue box for creating mipmaps on the main page).

Advantages of mipmaps

Mipmaps enhance the performance of your textures by lowering the amount of scaling that your computer has to do.

In addition, they remove a flickering effect of the pixels that can be observed when zooming in or out of a large texture.

Disadvantages of mipmaps

The only disadvantage of mipmaps is that they increase the size of the image file. However, the increase is only about 10%, so it is almost always worth it for the gains in performance.

Important

You should create the mip-maps before you convert the file to DDS; this results in files not much bigger than an ordinary DDS. Do not create mip-maps after the conversion to DDS: this results in huge, unwieldy files.

Can I convert lots of textures without having to do each one separately?

Yes. Some compression utilities permit batch compression. You can select numerous files in a folder and request that each be converted to .DDS. This obviously saves a lot of bother; for example it enables you to swiftly convert the hundreds of small images in a virtual texture.

The following describes how to do this using ATI's Compressonator (other utilities will work differently).

From the 'File' menu, select 'Batch Compress'. Find the folder in which your textures are stored. Select all the ones you want to compress (you can draw a box around them to highlight many files; or, you can click on files while holding down the 'Shift' key). Use the dialogue boxes at the bottom to choose a conversion to .DDS (click the 'Options' button to choose the required DXT format). Make sure you also choose the correct folder to save your converted files to. Then click 'Compress'.

How can I create and edit .DDS files using Windows?

You shouldn't. It's much better to create and edit files in another format and then convert to DDS as a final step.

However, if you really want to, your basic problem will be that the standard Windows tools don't enable you to edit DDS textures. You'll need to download new software.

XnView v1.80.3 allows you to write .DDS files. Download from this website: http://www.xnview.com/
Note: some Celestia users have found problems with this software - consult this thread: http://www.celestiaproject.net/forum/viewtopic.php?t=8067

PhotoShop and Paint Shop users: The NVidia website - http://developer.nvidia.com/object/nv_t ... tools.html - offers a plugin for Adobe PhotoShop and Jasc PaintShop Pro that allows you to edit and create .DDS files.

Gimp: There is not yet a Windows executable plugin for DDS.

Final Request
As stated above, this is a preliminary FAQ written by a novice for novices. If anyone wiser can add extra advice, please do so and I'll edit it in

[Harry]

It's better than the .JPG files that are provided with the standard Celestia installation.

You should rephrase that - they are not generally better, their main benefit is that they need less graphics card memory. Another difference is that DDS can include an Alpha channel and because the graphics card can use them without a complicated decompression step they also load much faster. But JPG provides better visual quality (except when you use a very low quality setting while saving) and needs less diskspace (except when you use extremely high quality settings).

Will .DDS files work on my graphics card?

Another way to check this is to get the list of supported OpenGL extension from Help->OpenGL Info, and check for GL_EXT_texture_compression_s3tc, which is needed to display DDS textures.

Harald

[The Singing Badger]

Thank you for your help, Harry, I appreciate it and I will rework the posting accordingly.

By the way, what exactly does it mean to say that JPG images provide better image quality - what's better about them, exactly?

Thanks again,

TSB

[hjw]

By the way, what exactly does it mean to say that JPG images provide better image quality - what's better about them, exactly?

One, maybe two reasons:

1. DXT1/3/5 compression is LOSSY:
1a) color-depth is reduced to 5-bit red, 6-bit green, 5-bit red (EDIT: 5-bit blue)
1b) for every square of 4x4 pixels a maximum of 4 diffent
colors is used to display ALL 16 pixels. Two of the 4 colors
are used as 'endpoints' and the other two are an interpolation
between the endpoints.

2. The compression program may be, well, simple-minded
e.g. the DevIL library is quite fast but does not choose the
'best' two endpoint colors. Nvidia's conversion tool does a MUCH
better job.

The 'quality' depends heavily on the input image. Smooth
color changes are handled better.

You NEVER EVER want to edit a DXT image!
Edit the input image which should be stored in a LOSSLESS
format (NOT jpg, NEVER DXT). Conversion to DXT should
always be the last step.

For RGB (without alpha) NEVER use DXT3 or DXT5. DXT1 is
half the size. DXT3 adds an 4-bit alpha channel (uncompressed)
and DXT5 adds an 8-bit alpha channel (compressed but lossy).

hjw

[The Singing Badger]

hjw,

Thank you very much for this information, I will rewrite the FAQ accordingly.

Two questions:

(1) Alpha channels are for textures with transparent sections, right? So a normal planet texture wouldn't need an alpha channel? But if I were converting a cloud map to DDS, would I need to use one of the other compression methods: DXT 3 or 5?

(2) You imply that the NVidia compression program is better quality but slower than DevIL and the ATI program - is this correct?

If anyone can confirm the above, Id' be grateful.

Thanks again,
TSB

[selden]

When an image file is specified in a Texture declaration, Celestia uses the Alpha channel of the surface texture image as the object's specularity. (e.g. for oceans on a planet)

When an image file is specified inside a model file, Celestia uses the Alpha channel of the surface texture image as the object's opacity. (e.g. for a Nebula)

When an image file is specified in a CloudMap declaration, Celestia uses the Alpha channel of the surface texture image as the clouds' opacity. [edit]Yes, you have to use either DXT3 or DXT5 format. DXT1 doesn't have an Alpha channel.[/edit]


I have not used either DevIL or ATI utilities, so I can't comment on them.

[hjw]

(2) You imply that the NVidia compression program is better quality but slower than DevIL and the ATI program - is this correct?

Yes. I implemented a S3TC DXT compressor/decompressor myself. I compared the
outputs of DevIL, NVidia and mine. Meanwhile I can achieve results comparable
to Nvidia's, but am still working on speed and DXT1a.

(1)
I don't have the source of NVidia's utility, but if you look at the source of devil
you can easily see that
a) depth scaling is not exact
b) the method used to compute the color endpoints is very primitive
c) alpha for DXT3/DXT5 is not used for rgb endpoint calculation, e.g. pixels
with alpha=0 (fully transparent) should be ignored, etc.

EDIT :
-------------------------
Well, the "alpha" channel is not always used as alpha channel (e.g. in normal maps).
Using (c) may be fatal in this case.
-------------------------

(2)
Depending on the input image the difference is VERY visible.

(3)
I used an analizer to compute the difference between input and output:
(r-in - r-out)^2 + (g-in - g-out)^2 + (b-in - b-out)^2 for each pixel.
This also shows that devil is inferior.

Edit:
BUT:
- nvidia has a limit to the texture size (8k or 16k I think)
- DevIL has no limit but loads the COMPLETE image into memory. This library
is designed to ALWAYS work on whole images, which is a problem with large
textures and little ram.


hjw

[The Singing Badger]

Thanks people, this is great stuff!

[hjw]

One more hint concerning BIG textures.

Problem:

On my system (SuSE 9.1, NVidia GF 6600 GT) big DDS textures
(4k and up) looked ugly when I MOVED through space or zoomed
in/out from some distance. The pixels "flickered" because
the graphics hardware does no smoothing when scaling down. I had this
effect also with my old GF2 MX 400. Fullscene anti-aliasing didn't help.

Solution:

Generate mip-maps (the nvidia tool does this by default, but
initially I turned it off to save space). The images get a little bit bigger,
but that's ok because the scene looks MUCH better when moving. It
took me quite some time to figure this out (might be obvious to experts).

I'm a linux-only user, so I run nvidia's windows software with "wine".
Don't know anything about ATI's stuff...

hjw

[maxim]

The images get a little bit bigger

Actually, adding all mipmap levels always increases pic/texture sizes from 3/3 (=1) to 4/3.

maxim

[hjw]

DXT3 saves the file with a compressed alpha channel, while DXT5 saves it with an uncompressed alpha channel (thus resulting in better quality but a larger file size).

- DXT3 has 4-bit UN-compressed, DXT5 has 8-bit COMpressed

- Filesizes: DXT3 has EXACTLY the same file size as DXT5. DXT1 is HALF the size
of DXT3/DXT5.

- DXT3/DXT5 does not improve RGB quality, only adds alpha. The choice between
DXT3/DXT5 depends on what you want to do (specularmaps/cloudmaps/normalmaps).

hjw

[The Singing Badger]

Hjw,

The choice between DXT3/DXT5 depends on what you want to do (specularmaps/cloudmaps/normalmaps).

What exactly is the choice here? Why might one choose to use DXT5 instead of 3? How would it affect a cloud map or normalmap, for example?

Thanks in advance,

TSB

[selden]

What exactly is the choice here? Why might one choose to use DXT5 instead of 3? How would it affect a cloud map or normalmap, for example?

- DXT3 has 4-bit UN-compressed, DXT5 has 8-bit COMpressed

So DXT3 alpha channels do not have as large a range of colors as DXT5 alpha channels. They won't be as smooth and will have more artifacts.

Also DXT5 textures support "mipmaps". The person creating them can insert several copies of the surface texture, each with different resolution -- each lower resolution image being half the width and half the height of the next higher resolution image. The graphics hardware will choose the image appropriate for the size of the object that it's drawing on the screen. This improves performace and appearance over a single mipmap since the hardware has to do less scaling.

[hjw]

What exactly is the choice here? Why might one choose to use DXT5 instead of 3? How would it affect a cloud map or normalmap, for example?

DXT compression has not been designed with normal-maps in mind. It's not easy to generate
GOOD normal maps with DXT.

What *I* do with *my* celestia installation:

- Textures (without alpha): DXT1

- Normalmaps: PNG or DXT5 (but I keep the PNG as alternate surface).

- Others: well, they are not *that* important to *me*, so I just use what gets installed.

- I always generate mipmaps when using DXT1/3/5. I strongly recommend this.

- In most cases I avoid JPG. What matters (to me) is the size in MEMORY, not on DISK.
A 3-channel JPG needs the same amount of memory as a 3-channel JPG. But PNG
is lossless.

- I try to get images as PNG, if this fails as TIFF, if this fails as JPG.

- If JPG is all I can get, and I want to manipulate it, I try ' jpegtran' to do LOSSLESS
transformations (e.g. crop a 1024x1024 rectangle out of a 1073x1758 image), skipping
the LOSSY decoding/encoding step.


Other celestia users surely have other preferences. The above is what *I* do.

hjw

[t00fri]

...
DXT compression has not been designed with normal-maps in mind. It's not easy to generate
GOOD normal maps with DXT.

Right. Normalmaps need maximal possible smoothness and minimal NOISE, since they involves calculation of gradients=normals...

I always start from the 16bit grayscale levelmap originals
and feed the RAw format into Chris nm16 program. This way all required image manipulations are done at the 16bit level per channel and only at the /very/ end, the result is truncated to 3x8bit RGB, as required by Celestia normal maps!

What *I* do with *my* celestia installation:

- Textures (without alpha): DXT1

- Normalmaps: PNG or DXT5 (but I keep the PNG as alternate surface).

That's precisely what I do since > 2.5 years with my Celestia
installation. But note that my graphcs card has 256MB DDR memory and my machine has 3GB of RAM.

...
- I always generate mipmaps when using DXT1/3/5. I strongly recommend this.

That's indeed an elementary lesson. Should always be used.
The savings in texture size are only ~10% if mipmap generation is skipped.

- If JPG is all I can get, and I want to manipulate it, I try ' jpegtran' to do LOSSLESS
transformations (e.g. crop a 1024x1024 rectangle out of a 1073x1758 image), skipping
the LOSSY decoding/encoding step.

Another proven possibility is to convert JPG's into lossless PNGs before any manipulations are performed...


Bye Fridger

[The Singing Badger]

Thank you everyone for your advice, I've edited it into the FAQ. I'm very grateful for your patience with my questions.

TSB

[TheStressPuppy]

It seems im having a very difficult time saving textures as DDS using the nvidia photoshop plugin. While i can save smaller textures with no issues (1024x512's) anything higher resolution like an 8k texture i get a "Could not save "C:\celestia\textures\hires\jupiter.dds" due to program error" which the program wont say what the error is. I tryed 2 different photoshop version (PS7 and CS9.0) on 2 different computers (a laptop, and a desktop), and got the same error on all of the above with anything above 1k. At 1st i was thinking it was a ram issue, but i have 2 gigs of ram on my desktop. Any thoughts or suggestions on what could be the problem? (besides 2 bad computers)

desktop specs... AMD sempron 3100+. 2 gigs pc-3200 ram. BFG-GF-6600gt 256 megs. asus k8n MB running winxp sp2. using adobe PS CS9.0. and nvidia photoshop dds plugins v

laptop pentium 3 700, 128 megs ram, crappy built in ati gpu (2 megs vid ram) basicly industrial art and all but useless cept 4 net surfing, but does run adobe 7.0

[selden]

Current 3D graphics hardware does not directly support the use of surface textures larger than 4K on a side. Inexpensive 3D graphics chipsets can only use textures that are 1K or less on a side. Celestia cuts larger textures into pieces for use on Celestia's internal spherical models. This sometimes introduces visual artifacts at the edges of the cuts that can be seen if you inspect Celestia's images very closely.

The authors of Nvidia's plugins have not tried to make them work for textures larger than what their hardware can support. Apparently there are cards being designed that will be able to use textures up to 8K on a side. Supposedly nvdxt can generate dds textures that large. To generate larger dds textures, you'll have to use some other utility. Some people have managed to use the DevIL graphics library to do this. See http://openil.sourceforge.net/projects.php

[TheStressPuppy]

Thank you for the quick reply.

I didnt think about hardware resolution limitations. That indeed explains alot. Also how do most of the artist here create 16k+ textures if hardware doesnt support it? While my 6600 may not be able to, Surely most newer SLI cards can more than handle a 16k texture.

Ill check out that program you linked and thanks again

[selden]

Thank you for the quick reply.

I didnt think about hardware resolution limitations. That indeed explains alot. Also how do most of the artist here create 16k+ textures if hardware doesnt support it?

The key word is "directly".

While my 6600 may not be able to, Surely most newer SLI cards can more than handle a 16k texture.

Nope, sorry. 4K is the current maximum for individual surface texture images directly supported by graphics hardware.

The large textures that people design are intended to cover an entire sphere. Celestia cuts those large textures into 1K, 2K, 4K or smaller pieces that will fit into the hardware and maps those pieces onto separate facets of its internal spherical models.

PNG and JPG images can be used without the size limitations of Nvidia's plugins. Of course, when they're expanded for use (DDS files don't have to be expanded), they're extremely large and you'll run into other limitations of computer hardware. Remember that an 8Kx4K image requires 128MB when fully expanded: 8x1024 x 4x1024 x4 8bit color+alpha channels. A 16K image requires 4x that: 512MB.