64kMars Map WIP

[DBrady]

I made a first attempt at actually building a 64k virtual texture from the MOC data.
Here are some results...


This is a view of the Noctis Labyrinth region. Some fine details can be seen in the canyons.

Next is a couple of valles Mariners...




Also looking quite detailed : )

Finally the happy face - somewhat of a recreation of a pic Jack Higgins posted in another thread.



I'll see about getting some more space sometime this month to host it.

There are many flaws in the texture however. The most prominent of which is a band of missing data around the south pole. There are also numerous seams through out the texture.

I'm not all that happy with the colour either. I think i'll try again on this aspect which does take a good deal of time too. Although i do feel its quite close to what the final colour might be. Its never going to be as red or even close to as red as the space graphics texture.

Also, quite dissapointingly, it doesnt match up with my normal maps. It looks to be 20 or 30 pixels out in some instances.

[RND]

wow those are impressive, Id love to be able to do stuff like that.

[maxim]

This is very nice work DBrady. I'd like to see the whole thing in the end.

But what's the matter with your color resolution? I've downloaded your FlatMars + additional stuff and found it very well color-balanced from the distance. But when zooming in, the colors divided into large monochromatic areas as if there was a color limit of 64 or 128 different ones - so I skiped your mars texture again.

Looking at the pics you've posted, I see the same effect again. Where does this originate? I haven'd seen that on other mars textures.

maxim

[dsroy]

DBrady: you're on the right track with your color - Mars simply isn't a red as the Space Graphics version. A pale salmon for the bright sands regions is becoming the current consensus, based on the most recent (and best) Hubble whole disk images, and hundreds of amateur astrophotographers.

My suggestion, is to keep the current coloration as is, and only concern yourself with the visible seams. Then, downscale your finished project to 16k or 8k and patch with some other polar region (matched to your color palette) to make a complementary normal texture.

I can only marvel at the patience you have in working with a texture (or tiles thereof) with 2.147 BILLION pixels. My computer choked for hours on hemispheres of a 16x8k regular texture. Hopefully you've created scripts that automate the work in the background.

[Buzz]

Beautiful! But I have to agree with Maxim: excellent bumps, but a limited amount of colours. Does your source material include more colours?

[DBrady]

Maxim and Buzz - limited colours:
This is actually one of the first things i noticed. While making the texture -mostly with scripts except colouring as i as yet do not know how to automate things with gimp - i noticed some very nice 'Dust cones' around alot of crators in the darker regions where lighter colour dust, blowing in dust storms, settled around each crator. An illustration will probably help...



I didnt post this picture before because of all the colour bleaching in the cones which does not appear at all in the uncompressed ppm files but only in the DXT1 version. These cones are most vived in ppm. There are also other instances of ejecta from recent impact crators which show up quite well in the ppm's.

At the moment i've come to the conclusion that its the DevIL image library(1.6.5) thats to blame for this. This is what i used for the conversion to dds as it can do it straight from ppm to dds and it's shear speed of course. As has been mentioned before this is quite buggy with dds formats but which i had thought was ok for dxt1. I did notice it in the flatmars texture i released before, but since that was only to provide a basic colour for the normal map, i didnt really look into it much as i didnt think it mattered alot because it wasnt providing much detail. Besides, i think Praesepe has a flat mars available too and any veteran celestian could have used that.

My next step was/is going to be a png or nvdxt DXT1 conversion of the 64k map to see how the colours faired in that. I'd also like to use DevIL 1.6.6 but i can't seem to find it on the DevIL site - it may be a cvs release - i must ask Fridger.

As for the overall colour, it really looks terrible on a global view - almost brown. I'll post a pic soon.

Buzz:
The original data is grayscale http://www.msss.com/mgcwg/mgm/
The lower resolution does not have the missing data around the south pole but seems to be corrected so it matches the Mola data and hence doesnt match up with the higher res data. Patching with it is therefore not an option without alot of fidddling or patching with any other texture for that matter.

And as for the seams, i don't think, or actually i know, i dont have the patience to fix them. Look how long its taking Don Edwards to make his 32k earth! This is like four 32k textures stuck together!

[danielj]

Why are you trying to build a 64 k VT texture,if 16k texture didn?t work in my computer.You have to improve the 16k texture VT.When I unzip with Winzip,it don?t go in separate levels.I had to do this manually,which was almost impossible.Anyway,I though I had to download levels 0,1,2 and 3 to work(all 4 zip textures),is it true?If so,even a 16 k VT texture is a monster file...

I made a first attempt at actually building a 64k virtual texture from the MOC data.
Here are some results...


This is a view of the Noctis Labyrinth region. Some fine details can be seen in the canyons.

Next is a couple of valles Mariners...




Also looking quite detailed : )

Finally the happy face - somewhat of a recreation of a pic Jack Higgins posted in another thread.



I'll see about getting some more space sometime this month to host it.

There are many flaws in the texture however. The most prominent of which is a band of missing data around the south pole. There are also numerous seams through out the texture.

I'm not all that happy with the colour either. I think i'll try again on this aspect which does take a good deal of time too. Although i do feel its quite close to what the final colour might be. Its never going to be as red or even close to as red as the space graphics texture.

Also, quite dissapointingly, it doesnt match up with my normal maps. It looks to be 20 or 30 pixels out in some instances.

[maxim]

Ugh... danielj could you please edit your quote text, so that the picture repetition is removed. Thanx

maxim

[maxim]

At the moment i've come to the conclusion that its the DevIL image library(1.6.5) thats to blame for this. This is what i used for the conversion to dds as it can do it straight from ppm to dds and it's shear speed of course.

...

My next step was/is going to be a png or nvdxt DXT1 conversion of the 64k map to see how the colours faired in that.

Does this mean that you are actually working on the dds-converted pictures? Bad thing if the answer is yes - that would mean you lose quality even before starting to work on the pics - same for png.

Another thing I have to complain about is that you stored your tiles of FlatMars and MarsNormals with MipMaps. MipMaps are completely useless in VTs but increase your filesize by 4/3 of the original. You should correct that - it would lower download times and storage space.

maxim

[Guest]

that would mean you lose quality even before starting to work on the pics - same for png

PNG is a lossless format, you should loose nothing using it. Except perhaps if you convert between color spaces.

[maxim]

... Except perhaps if you convert between color spaces.

Thats what I've meant. It's restricted to 256 colors as I know it. Sorry if that wasn't pointed out.

maxim.

[Guest]

Most software supports PNG in both 8 bit and 24 bit modes. I believe the format can support any bit depth though. PNG is RGB however, so I think certain conversions such as from CMYK sometimes result in data loss which is what I was referring to.

[maxim]

DBrady,

I did some investigation on your Mars addons and found the following:

Saving your FlatMars Distibution without MipMaps reduced the whole VT from 113.8MBytes to 85.2MBytes with each tiles size reduced to 129kB. Removing the additional (and unused) AlphaChannel from your MarsNormals Distribution and saving without MipMaps reduced the whole VT from 905MBytes to 340MBytes with each tiles size reduced to 2049kB.
So the whole thing was reduced from 1,18GBytes to 425MBytes saving lot of disk space. Further on the loading, accessing and navigating in Celestia did remarkably speed up.

I've also took a look on the color distribution in FlatMars: Examining the histogram shows several distinct color peaks with heavy reduced color weight inbetween. Pushing up brightness and contrast to the limits shows, that there are quite several color shades available, but anyhow they are almost merged to distict groups around one peak color with others having only a minimal different shade from that. This indicates, that there propably was a stage of color adjustion before, which used brightness and contrast settings for adjusting, instead of changing gamma values, color balance or level balance. If this is not inherited by the source images, you may examine your working process for that.

Hope I could give you some hints.

Greetings, maxim

[DBrady]

Maxim,
I wasnt aware that mipmaps could be left out at all. In fact i thought they were needed for the new VT code as evidenced by a short exchange with Chris while he was developing the code(at least i thinkt it was part of the VT code)...

I'm working on some the new texture code and tried using a 16k texture that Don generated with your tool. It revealed a bug in my code, but I think that there's also a problem with the texture . . . It's 16x8k, so there should be 15 mip levels. But, the texture actually contains only 14 . . . My new code will not use mipmaps at all if the mipmap set is incomplete; could you check to see if you're generating the right number of mipmap levels?

His final outcome was...

I think that I have a reasonable workaround; if there are missing mipmap levels, I'll just have them point to the lowest level that was provided. At the very bottom end of the mip chain, there should be no visible difference.

Looking back this probably implied that mipmaps were not really needed at all but i never caught that.

None the less thanks for the valuable information. Leaving them in is really a blunder on my part.

As for the alpha channel, how did you remove this?
I thought it was always present in dxt3 format.
Did you just convert it to dxt1?If so is this not more compressed then dxt3 and hence destroys the normals more?

As for the colours of the flat mars map - i still think its to do with the dxt format. Have you looked at jims post http://celestiaproject.net/forum/viewtopic.php ... highlight=
It's very informative and also shows a bias away from the red channel for nvdxt and green for devil which i used. Have you compared the histograms from the original from space graphics and mine?

As for the optimised textures you made feel free to release them. Just the other day i deleted the full 46k ppm mola normal map from my hard drive. It was ~3-4GB and i needed space for the uncompressed 64k Moc files. Hence making another set of tiles is not so simple.

You do seem to have a better grasp of colours then me. I'd like to see what colour you could come up with colourising the Moc data. John van Vliet kindly made a 4k map available of the Moc data. If you could colourize that and take note of what you do i'd be happy to try and implement it in the 64k map.

[wcomer]

Actually mipmaps do make a difference with VT's. But the difference maynot be worth the bandwidth/storage tradeoffs.

Without mipmaps the texture tends to "twinkle" near the horizon, this goes away with the use of mipmaps.

-Walton

[maxim]

DBrady,
wcomer,

Relating mipmaps:
I know the theory for using mipmaps for near horizont rendering. But I thought that would be valuable primary for flat horizons and textures with high contrasts. I've never watched twinkling effects in celestia, but I would like to see examples on that. I think will leave mipmaps off (in VTs) until I'm convinced that the're truely needed - perhaps only for certain types for textures - and worth the overhead.

Relating alpha channel:
I had loaded some of your Normals tiles into Photoshop and discovered that the Normals where composed from the R,G an B channel, and the Alpha channel stayed empty - so I decided to remove that channel and save them with dxt1 compression. I don't know of how much more artifacts are produced by that compression, but I watched no difference in Celestia - at least no one I could remember from one start of Celestia to the next. So I'll leave that too, until someone's got a better reason for 600MBytes more data on disk.

Have you compared the histograms from the original from space graphics and mine?

No. I would do if I'd knew where to get the original data from.

As for the optimised textures you made feel free to release them.

Until now I have only removed some the data and I have do be carefull with my online space. So I won't do that until there's a remarkable amout of people asking for it.

John van Vliet kindly made a 4k map available of the Moc data. If you could colourize that and take note of what you do i'd be happy to try and implement it in the 64k map.

I could give it a try if I'll get a source link.

greetings,

maxim

[Buzz]

Maxim,

Using dxt compression of any kind with normalmaps results in horrible artefacts (squares), I am sure you will see them when you zoom in a little....

[wcomer]

I'd like to add slightly to Buzz's statement.

To see the artifacts: sync orbit, zoom in, observe dusk or dawn under high speed conditions. The squares should be painfully obvious.

Walton

[maxim]

Thanx both,

I'll take a look on it just before weekend.

Using dxt compression of any kind with normalmaps results in horrible artefacts (squares)

That means DXT1 or DXT3 or DTX5 are all 'bad' - so it doesn't make things more worse having DXT1 instead of 3?

What would be the preferable format?

maxim

[jim]

That means DXT1 or DXT3 or DTX5 are all 'bad' - so it doesn't make things more worse having DXT1 instead of 3?

What would be the preferable format?

Maxim, it does make things more worse when useing DXT1 instead of DXT3 compression - but only on Nvidea hardware. This is a really ugly bug on Nvidea cards with DXT1 compression that causes that only a reduced number off colors is used. But general there is no difference of DXT1 and DXT3/5 color compression only an alpha channel was added to DXT3/5 and double the file size.

Bye Jens