Playing with my new 16k DXT1 NormalMap!

[t00fri]

I have finally managed to produce a 16k DXT1c /normalmap/ of
earth. Although it is not yet a /true/ 16k map, but 'blown-up' from
the SpaceGraphics 10k bumpmap, the results with my new custom 16k earth
texture are truely amazing and much more detailed than with my
standard 8k normalmap. Here is a little taste from a north
polar region whose atmosphere I particularly like ...




The making of the 16k normalmap was a little tricky, given that I only
have 512MB RAM. I cropped the 16k x 8k bump file into two RGB 8k x 8k
pieces that I was able to convert each to a normalmap by means of the
new GIMP normalmap-plugin. The resulting normalmap pieces were then
merged with a simple Gimp-Perl plugin, called horiz-cat into the final
16k x 8k normalmap. This 80MB RGB *.tga texture was then converted to DXT1
by means of my own 'texconvert' tool that I wrote with the help of the
DevIL library. It works very fast and easily converts 16k textures to DXT.

That's it...;-). Oh, yes, with my 32MB GeForce 2 (!), including
clouds, spec and night lights, I still get about 15 fps/16bit color,
using both the 16k main texture and the 16k normal map!

And if Chris /soon/ finds a way to reactivate the CGI-code of my
Texture Foundry site, I can manage to transfer that stuff up
while I am still on holidays...

Bye Fridger

PS: ... And should you ever be fed up with 'generic' holiday resorts like
Acapulco;-), I can strongly recommend to travel with Eskimo fishermen
along the coast from Jacobshavn (see 16k image below;-)) up north to
Thule (81 degN!)...

[Psykotik]

It looks very nice Fridger, and since I own the same graphic card, I'm waiting for your site's reactivation...

Meanwhile, could you post a picture of the oceans ? I'm very curious to see what kind of colour do you get.

[t00fri]

It looks very nice Fridger, and since I own the same graphic card, I'm waiting for your site's reactivation... :wink:

Meanwhile, could you post a picture of the oceans ? I'm very curious to see what kind of colour do you get.

My new 16k earth texture is the first 16k texture where I have managed to match the colors almost perfectly to those of the SpaceGraphics 8k Render earth. Switching between 8k and 16k is only perceptible through the much improved resolution, NOT by the color.

An important trick was to merge the 16k dds file (without mipmaps) with the 8k SG dds texture including 13 mipmaps. The result is a 16k texture with 14 mipmaps and perfect color matching to SG.

Bye Fridger

[jim]

Hi Fridger,

I thought you are a week on Easter holidays?

My new 16k earth texture is the first 16k texture where I have managed to match the colors almost perfectly to those of the SpaceGraphics 8k Render earth. Switching between 8k and 16k is only perceptible through the much improved resolution, NOT by the color.

The SpaceGraphics Render earth looks good in any render programm but not in Celestia. I think that Celestia looks better with a brighter textures than the SpaceGraphics one. I've built a 8k Earth based on the Blue marble texture. The SpaceGraphics texture has also some bugs caused by a wrong water mask.

An important trick was to merge the 16k dds file (without mipmaps) with the 8k SG dds texture including 13 mipmaps. The result is a 16k texture with 14 mipmaps and perfect color matching to SG.

The mip maps offers some interesting possibilities unknow by the most peoples. There is a easy way to build a complete resolution map set (4k, 2K or 1k) from a 8k or 16k DDS file. The other way is to build a large DDS file without mip maps and combine these with a fitting smaller one with mip maps. But i can't unterstand why you merge your 16k texture with the 8k SG one the result will never be perfect. Why did you not build all the mip maps with Gimp and save this in one folder. Then you start a batch process and build 14 DDS files without mip maps which are finally merged with the NV tools to a 16K DDS file with mip maps.
This methode has same advantages:
1. better quality of the mip maps (a photo editor has a better resize funktion then the nvdxt)
2. nvdxt works faster and needs less resources
3. no problems with differnd textures (your 16k and the 8k SG one)
4. several created mip maps promise a better quality particular for normal maps

Ok, now another point you told about a DDS conversion tool form 'Pixel'. I searched the forum but can't find the tool. Is it possible that you send me this tools via mail.

Here is the command I use, <file.tga> is RGBA! <====

nvdxt -file <file.tga> -alpha -n4 -scale 10.0 -dither -wrap -24 dxt1c

It may actually be a bug in the tools, that the alpha channel is still carried along as a 'dummy' channel, since the starting file was RGBA. I look into that.

All 16k conversions I usually do with my own tools, since they are much faster than the NVIDIA ones. Typically less than a minute (!) for *.tga => DXT1.

I told you also that I am using the mipmaps from the 8k SG image together with a 16k converted DXT texture that does not have mipmaps! And so on...

Did you use /exactly/ the same procedure???
Otherwise your results may be quite different.

I know this way to make a normal map and have it already tested. My way is different a build the normal maps in Photoshop and save this to TGA or save they direct with the DDS plugin. The plugin works faster than the command line tools.

There is a little bug in your command line. Don't use the '-warp' option. Try to build a normal map form my test bump map or the Mars bump map and you will see a ugly line at the texture border with '-wrap'.

"Frohe Ostern"

Jens

p.s. The forum is terrible slow today for me.

[t00fri]

box is too slow...

[t00fri]

Hi Fridger,

The SpaceGraphics Render earth looks good in any render programm but not in Celestia. I think that Celestia looks better with a brighter textures than the SpaceGraphics one. I've built a 8k Earth based on the Blue marble texture. The SpaceGraphics texture has also some bugs caused by a wrong water mask.

Of course, it is most crucial to have the appropriate cloud layer with a bluish profile on top of SG earth. Thats also what was behind their 'Render' coloring. I do not want to enter this old discussion again...

My earth texture certainly looks good to my personal standards and has also been compared with the best Satellite photos, of course;-)

The mip maps offers some interesting possibilities unknow by the most peoples. There is a easy way to build a complete resolution map set (4k, 2K or 1k) from a 8k or 16k DDS file. The other way is to build a large DDS file without mip maps and combine these with a fitting smaller one with mip maps. But i can't unterstand why you merge your 16k texture with the 8k SG one the result will never be perfect. Why did you not build all the mip maps with Gimp and save this in one folder. Then you start a batch process and build 14 DDS files without mip maps which are finally merged with the NV tools to a 16K DDS file with mip maps.

Since my whole color scheme is systematically based on the one by SpaceGraphics, I wanted to have a /perfect/ colour matching when switching resolution. Also I am somewhat short of spare time;-). I do not see what you mean: I did generate /all mipmaps/ for the 8k set myself using my /own tool/ which employs B-spline resizing. SG just provided a 10k JPG texture, no mipmaps, of course. I just use exactly the same mipmaps both in the 8k dds and the 16k dds textures.
That's all and there is no better method...

This methode has same advantages:
1. better quality of the mip maps (a photo editor has a better resize funktion then the nvdxt)
2. nvdxt works faster and needs less resources
3. no problems with differnd textures (your 16k and the 8k SG one)
4. several created mip maps promise a better quality particular for normal maps

I do not understand any of your points 1.-4.

Ok, now another point you told about a DDS conversion tool form 'Pixel'. I searched the forum but can't find the tool. Is it possible that you send me this tools via mail.

If they are not too long, I'll see. They have never been public.

I know this way to make a normal map and have it already tested. My way is different a build the normal maps in Photoshop and save this to TGA or save they direct with the DDS plugin. The plugin works faster than the command line tools.

Doug says in agreement with me that nvdxt is faster than the photoshop plugin. The code is the same anyhow, photoshop needs a lot of additional memory...


Bye Fridger

[jim]

Hi Fridger,

The SpaceGraphics Render earth looks good in any render programm but not in Celestia. I think that Celestia looks better with a brighter textures than the SpaceGraphics one. I've built a 8k Earth based on the Blue marble texture. The SpaceGraphics texture has also some bugs caused by a wrong water mask.

Of course, it is most crucial to have the appropriate cloud layer with a bluish profile on top of SG earth. Thats also what was behind their 'Render' coloring. I do not want to enter this old discussion again...

I know that this was already detailed discussed and that the clouds and the haze effect must work together with the earth map to get perfect colors. But I think you know that Celestia can't render things brighter that means a texture must be desinged with the maximum brightness of the Sun.

My earth texture certainly looks good to my personal standards and has also been compared with the best Satellite photos, of course;-)

I do not want criticise your texture but i think different people have different ideas about the perfect earth texture;-)
You are not alone comparing Satellite photos. I have uploaded enough pictures that I can say it's hard to find a photo which is good exposed and with true colors (not edited, color optimized or something else).

This methode has same advantages:
1. better quality of the mip maps (a photo editor has a better resize funktion then the nvdxt)
2. nvdxt works faster and needs less resources
3. no problems with differnd textures (your 16k and the 8k SG one)
4. several created mip maps promise a better quality particular for normal maps

I do not understand any of your points 1.-4.

Point 1: I know from the mip map filtering options but if they are used the conversion time is unacceptable.

Point 2: If nvdxt has no mip maps to create it works fast and needs less resources. I've made a little BAT-file with nvdxt, stitch and build the mip maps with an image editor. By this way the conversion time was halved.

Point 3: I find it more difficult to adapt a texture pefect to existing mip maps than to build new mip maps. But i unterstand you want that your new 16k map has the same colors like the old 8k texture.

Point 4: I think a downsized normal map lose the bump effect and it's better to build the first mip maps several from resized bump maps.

Doug says in agreement with me that nvdxt is faster than the photoshop plugin. The code is the same anyhow, photoshop needs a lot of additional memory...

This was also a suprise for me that the photoshop plugin is faster. It needs round 15 minutes to build a 8k DXT3 file while NVDXT needs 30 minutes. And if I use some special options (e.g. Full DFT filter or Normalize Mip Maps) NVDXT needs up to some hours.

OK before you ask:
My system: Win98SE, duron900, 512MB, geforce3, max. 8GB swap file

Please tell me when you have found Pixel's conversion tool.

Of topic: I've seen a tv report in 'Nano' about desy in Hamburg. Was this about your job?

Bye Jens

[Pixel]

Hi Fridger,

I have a true 16K EearthBump texture, based on 1km-elevation GLOBE project: http://www.ngdc.noaa.gov/seg/topo/gltiles.shtml

the bump-map on SpaceGraphics is also based on the same data, but it is only 10K and also has been postprocessed - i think unshapr filter has been used, or something that make the image look very rough.

So I have reduced the 43K 16-bit elevation data to 16K 8-bit gray-scale jpeg.
You may consider to use this bumpmap for your normal maps. It is only 2.9 Mb, but i am still looking where to upload.

[t00fri]

Hi Fridger,

I have a true 16K EearthBump texture, based on 1km-elevation GLOBE project: http://www.ngdc.noaa.gov/seg/topo/gltiles.shtml

the bump-map on SpaceGraphics is also based on the same data, but it is only 10K and also has been postprocessed - i think unshapr filter has been used, or something that make the image look very rough.

So I have reduced the 43K 16-bit elevation data to 16K 8-bit gray-scale jpeg.
You may consider to use this bumpmap for your normal maps. It is only 2.9 Mb, but i am still looking where to upload.

Hey Pixel,

welcome back! Where in this world are you presently floating?;-) Come on, we need your expertise....

I am /most/ interested to put my hands on your 16k elevation map! I only find it a pity that you did not leave it at 16bit, since this is quite crucial for smoothness (absence of artifacts in normal maps).

Can't you put the file to Smirnov's account. It's still there at the old address. So I could right away load it down.

It is sort of urgent, since I wanted to make use of my spare time over the weekend and prepare the new textures for a thorough upgrade of my Texture Foundry site.

Perhaps also Christophe (Teyssier) can offer you some temp WEB space for uploading...

Of course I am aware that my 16k map is only a compromise, being generated by blowing up the SpaceGraphics 10K bump map. But still it is quite good already.

Bye Fridger

[t00fri]

After some further interaction among the developers, Chris managed to fix the bump/normal map bugs still inherent in 1.3.0 once for all!

I have just been conducting a number of tests and am convinced that this is a great improvement.

Below you see the correct light/shadow display for the sun position over the Andes. I always use an "empty" brownish topography texture along with a 16k normal map and the spec lighting texture to make the sun's position more apparent:



Note also: The familiar "altitude inversion" effect on Mars is also essentially gone now or much alleviated. It was largely due to left over bugs (mainly apparent in the olympic mons region) in the bump code and not or barely so due to "preshading" of the topography map!

Bye Fridger

[Guest]

Thanks Fridger,

I have no much to offer that can justify my own web space here. A few strong addicted celestrians like you, are enough .
Smirnof's ftp site seems don't work for me, so i will try to email you the file (2.9Mb). I'm also wondering how to preserve the 16bit source accuracy in order to create nice normal map, but I have no idea. There is no such tning as 16bit grayscale JPEG! Nor I can find any graphicgs file format that supports 16bit single-channel data. SpaceGraphics file is also 8-bit. If you have idea tell me.

have a nice weekend
Pixel.

[t00fri]

Thanks Fridger,

I have no much to offer that can justify my own web space here. A few strong addicted celestrians like you, are enough ;).
Smirnof's ftp site seems don't work for me, so i will try to email you the file (2.9Mb). I'm also wondering how to preserve the 16bit source accuracy in order to create nice normal map, but I have no idea. There is no such tning as 16bit grayscale JPEG! Nor I can find any graphicgs file format that supports 16bit single-channel data. SpaceGraphics file is also 8-bit. If you have idea tell me.

have a nice weekend
Pixel.

Hey Eugene,

well, your 16k levelmap safely arrived by mail. Thanks a lot! I now have converted it first with a GIMP-plugin to a normal map and then with my own tool (that can also read *.raw format!) to DXT1. The texture is certainly good, but unfortunately, as I suspected, due to the 8bit limitation does not provide this "final kick";-).

You can simply see from the normal map that 256 graduations are by far inadequate (divide the altitude of Mt.Everest by 256, then you know what this file calls "flat"). My blown up 16K normal map is doing much better, in fact. It is 16bit throughout. In fact the SpaceGraphics bump JPG is said to be 16bit by them. The SG Mars bumpmap is only 8bit, though.

Why did you not convert the 16bit GLOBE raw data via GIMP or Photoshop to *.tga, say?


Bye Fridger