Cube maps

[chris]

Correct, but on the other hand, the pole singularities are confined to the environments of 1 point each! The maps of the many edge and corner singularities are spreading over many regions of the sphere where we usually want to render more concise information than at the poles...

The singularities are a single point, but in practice the artifacts are visible over a large area near the pole. I'm honestly not sure why you're defending simple cylindrical projections--they have obvious inadequacies when compared to cube maps, as the mapping from sphere to cube map is well-defined everywhere.

Could you please indicate whether you can see here any kind of 'pinch' problem? This display is just the result from my RGBA base+spec tiles obtained with my new txtools, together with my normalmap tiles from the nmtools. This would clearly be MUCH worse in case of a "one piece" cylindrical texture. "Pole-optimized" VT's really do a good job. I have studied this in great detail in the past.

You've chosen a best-case scenario for the simple cylindrical projection. The near uniform white in Earth's polar region is advantageous for hiding artifacts in that area. With the Moon and most other bodies, we're not so lucky, and I maintain that cube maps would be obviously superior. Also, the area affected by the polar degeneracy shrinks as the size of the texture increases, and there aren't many 64k VTs out there.

And again, there are my original motivations of cloud shadows and cloud textures, where cubemaps have big advantages.

The true test of course is to actually generate and use a cube map in Celestia. I'm confident that it will produce a noticeably superior rendering at the poles, without noticeable artifacts elsewhere.

--Chris

[t00fri]

Could you please indicate whether you can see here any kind of 'pinch' problem? This display is just the result from my RGBA base+spec tiles obtained with my new txtools, together with my normalmap tiles from the nmtools. This would clearly be MUCH worse in case of a "one piece" cylindrical texture. "Pole-optimized" VT's really do a good job. I have studied this in great detail in the past.

The near uniform white in Earth's polar region is advantageous for hiding artifacts in that area. With the Moon and most other bodies, we're not so lucky, and I maintain that cube maps would be obviously superior. And again, there are my original motivations of cloud shadows and cloud textures, where cubemaps have big advantages.

OK, here are my Mars tiles near around the pole....
They are NOT uniform white, are they?



See anything that would justify the development of new cross-platform software to make "cubemap" texture creators happy etc?

And by the way, there are not so many planets where cloud textures are of importance

The true test of course is to actually generate and use a cube map in Celestia. I'm confident that it will produce a noticeably superior rendering at the poles, without noticeable artifacts elsewhere.

--Chris

NO doubt about this

Bye Fridger

[ElChristou]

OK, here are my Mars tiles near around the pole....
They are NOT uniform white, are they?...

Huumm... quite impressive indeed...

[t00fri]

I'm honestly not sure why you're defending simple cylindrical projections--they have obvious inadequacies when compared to cube maps, as the mapping from sphere to cube map is well-defined everywhere.
....
--Chris

Of course, I am aware of the deficiencies of simple cylindrical projections. But I am defending them for a number of good reasons, I think:

-- In Celestia, we do have a number of "quickly coded features" already that are not terribly useful in the forseeable future.

In case of cubemaps, I think the danger is similar, if you just add some lines of code and then state "as of now Celestia supports cubemap textures".

I think to render cubemaps really useful, a substantial cross-platform development of "infrastructure" (support software!) is required as well! This however will not happen JUST for supporting 8K textures a bit better near the poles and to improve on cloud texture size.

Notably all 8k moon textures from recent space missions show low resolution areas and other deficiencies. Suboptimally rendered pole environments can easily be taken care of with standard pinch filters, too, and would really not stick out among the other imperfections...

I would never sit down developing a set of respective tools knowing from the onset that the biggest useful texture sizes would be 8k or at the very best 16k! Not to speak of possible deterioration of texture quality in other regimes of the body. It has not been proven yet that this concern is void!

It is a fact that the imaging teams of space missions do NOT publish ready-to-use cubemap texture sets of their scientific imaging data. So their output WILL NEED substantial non-trivial reprocessing by someone, in order to be useful in Celestia. We have not heard any convincing plans from your side wrto this crucial issue.

-- on the other hand we do have a set of tools NOW that even unexperienced users have successfully used on their home-machines, allowing to generate arbitrary, even monster sized, "pole-optimized" VT tile sets with a single click in all supported OS.

++++++++++++++++++
I think it is just not a good policiy if you are injecting yet another "half-finished" concept at this stage. We could use the required menpower much more profitably for really lacking things.
++++++++++++++++++

That's about my reasons for being so "naive" as to supporting simple cylindrical textures, --if useful-- optimized in form of tiles.

Bye Fridger

[ElChristou]

Fridger, in your case (optimal) I agree (seing your shots) that cylindrical VT maps are fine enough(!) to not worry about anything else.

Now, for the basic end user (I mean for the basic package of Celestia) I suppose cube maps would be a huge improvement comparated to the actual default maps.

Don't forget many people over the word don't have yet the possibilities of occidental countries (dsl at low cost), it's why for example from here (Paraguay) I still haven't test your tools simply because I'm unable to download the necessary datas for processing.

Also, I'm not sure educators will really take the time to process all those datas (I mean download, then process), they will use the default package or what is available ready for use...

Perhaps the ideal would be a code to chose what kind of projection you want to use on a particular body. If this is possible, a string in the ssc would determinate this before the string calling the texture file (or set).

A question: from your top 64 or 32k textures, is it possible to get a 2k without pinch effect?

[t00fri]

Christophe,

Fridger, in your case (optimal) I agree (seing your shots) that cylindrical VT maps are fine enough(!) to not worry about anything else.

Now, for the basic end user (I mean for the basic package of Celestia) I suppose cube maps would be a huge improvement comparated to the actual default maps.

But who will undertake the TEDIOUS PART of making all those cubemaps? What do you think? Chris? With which software? No support for MACs, only Windows, of course, etc...

++++++++++++++
Unlike simple cylindrical projections, cubemaps are not a default publishing format of scientific servers!
++++++++++++++

Don't forget this. So it will boil down to a considerable /remapping/ task from mostly simple cylindrical format to cubemap before this feature can be enjoyed in Celestia.

That map is SINGULAR and needs considerable smoothing/filtering already. So in principle it would be enormously preferable to add the polar texture parts from special polar projections that sometimes are published. However then we are at a COMPLEX image manipulation task for EACH celestial body in question...

I have done this kind of tedious reprojection & alignment job MANY times with my Titan, Iapetus,... textures. I know what I am talking about!

Who is going to invest the considerable effort for comparably little benefit?? I won't, for example. Not for doing only 4k-8k 'one-piece' textures.

A question: from your top 64 or 32k textures, is it possible to get a 2k without pinch effect?

The point is --as I explained above-- that the optimized reduction of the resolution in the polar regime is the main reason why the pinch problem is considerably alleviated in case of VT's! So if you don't use VT's, you get a pinch to start. However, you may use a 'pinch' filter or the polar map tricks e.g. in PS. For such small textures I would always use my own 'image manipulation magic' to get rid of pinched displays. That's NO problem, really. And it's much easier to handle than going all the way of first producing cubemaps from cylindrical maps etc.

Think instead of what happens with the original,undistorted image information through this set of maps: starting from simple cylindrical data one first has to perform a partly SINGULAR map to the 6 cubemap faces. Then one does the second step in Celestia, mapping the 6 faces with their edges and corners onto a spherical body. In each of the steps heavy filtering is necessary to suppress respective artefacts.

Chris asserts that this chain of massive filterings will remain invisible, such that effectively NONE of the subtle image information will be lost...

Let's wait and see

So far my understanding was that one of the essential reasons for the popularity of cubemaps in games was SPEED, since the required filterings can be done at the hardware level. But for games /truth of mappings/ is of secondary importance...

Cheers,
Fridger

[ElChristou]

Fridger, I agree with all your points from those technical point of view; I was only saying that better maps for the default package would be welcome whatever system is used.

It would be cool if we go for VTs to have let's say level 1-2-3 in the default package and the rest in some sets of official VT to download separatly...

This way Celestia would be much more attractive for the first time user, don't you think?

The idea of a hires official pack has been already discuss and everybody was agree to do it. Perhaps it's not easy to do it at once (maps + models) so what about begining little by little, one map after the other, one model after the other.
Those officials maps/models could be released in a page of the official site and the day there is enough items, we could pack them together...

Is this unrealistic?

[cyber_space_doc]

I should express some opinions

1) The VT's probably look fine at a high resolution.

2) I don't think replacing the VT's with cubemaps would be sensible to a user, intstead an extra option would be an idea. It is obviosuly too much hassle to try to convert the spherical textures into cubemaps or 6 seperate textures.

3) The "cubemaps" would be best applied to a geometric form consisting of 6 quad grids mapped to a sphere. (possibly a cmod?).

4) the cubemaps would need to be divided up if they where to be stored as quad-tree or cubemap VT's, and therefore they would not be proper cubemaps since cubemaps are of limited size.

5) The cubemap idea is actually quite good for heightmap representation, providing an easy to implement continuous progression from bumpy terrain to a texture map from space.
Solutions for spherical textures are also available, such as Spherical Clipmapping.

http://www.zib.de/clasen/?page_id=6

Both forms could probably progress as a smooth transition from a mixture of normal mapping and parrallax bumpmapping to real bumped geometry. Overall I think suport for hardware optimized heightmaps would be an excellent feature to add to Celestia.

6) Overall I think the creation of cube textures is a lot more diffucult than the creation of spherical texutures. It might be a cool feature to support cubic noise textures and also POV Ray style volume noise textures on the offchance that one technique might be more efficient than another in certain situations...

I have written most of a program based on the last one that shows the limitations and strengths of the cubemap approach with the possibillity of comparison to the spherical texture approach which I can post up in a couple of days once I have tidied up the code.

[edit] I was also thinking it would be useful to capture a cubemap of a space scene similar to taking a screenshot. It might also be interesting to work out a method of guessing a users position in space from a cubemap picture, as a solution lost-in-space scenario.

[cyber_space_doc]

Hi,

I have finally fixed up my demo. It is available at:

http://www.kai-soft.co.uk/noise3.zip

as before it builds with vc8 and direct3d on windows only. The new version is set to load textures instead of generate them. I have hosted large 2048 sized textures at:-

http://www.kai-soft.co.uk/Diffuse_2048.zip

textures to be pasted into the Diffuse folder such that the contents of the folder contains the textures. The app will not work without the textures.

http://www.kai-soft.co.uk/spec_2048.zip

textures to be pasted into the spec folder.

For people who would rather generate the textures either define
GENERATE_MAPS in the code or open the solution properties in vc8 and define GENERATE_MAPS as shown in the following picture:-



and rebuild. The .png images *must* be removed from the texture directories before it will load the new bitmaps.

The app uses a *much* more optimized cubemap texture generator. The original version had floating point precesion problems due to the zig-zag scanlines that I used to increment the 3d vectors used to index the noise map. I now use integer positions and double precision to calculate the noise map.

Press 'W' for wireframe, left mouse (with motion) to rotate and right mouse to zoom.

otherwise, for people who don't intend to download here is a screenshot of the app in action:-



and here is the north pole



here is a wireframe:-



here is a shot of the wireframe of the problem area:-



and here is a shot of the same area without wireframe



[/url]

[cyber_space_doc]

Fixed...

The program "window.exe" in the directory was the old version - it had to be recompiled before it would work. I also changed the project settings to output the program to the root directory when it is in release build configuration.

I would advise against trying to run it with 4k * 4k textures, since I completely filled up my Video Ram with these textures. They will also eat a large amount of hard drive space.

In the future I am going to make a jovian texture builder, and I am also going to write an algorithm for generating spherical texture VT's.

[cyber_space_doc]

ok, I suppose I'll take that as a firm *no*.

my atttude is that:-

"if it ain't broke, don't fix it"

however I can honestly say that I can hardly stand seeing the planet surfaces in Celestia if they are all going to be pinched and warped at the poles...

The Virtual textures do not run on my machine - the system will start to *thrash* the textures.
see:

http://en.wikipedia.org/wiki/Thrash_(computer_science)

This means that the system has to swap massive textures between the graphics memory and the RAM. The thrashing is caused by normal movements in Celestia, such as rotating and zooming.

worth it

was it worth it for me? Yes. I enjoyed writing the software. I would find it very easy to incorporate it into Celestia since I have proffesional experience with large programs. Anyone following my posts would probably think of me as *unqualified* if I did not say that.

btw:-

I think its a shame that no-one can be bothered to even comment on my program ... what is the problem with supporting multiple projection modes? I expect the cubemap format solves the age old question about *how could a sattelite capture a planet texture quickly?*. It is obvious that NVidia, ATI and Microsoft would not have used cubemaps to represent skydomes, reflections and shadows in all their best demonstration programs unless cubemaps looked better - they would have used the cylindrical projection instead.

[cyber_space_doc]

btw:-

the pictures I posted are egg-shaped.

This means that they are in *Wide Screen* mode and the squares will look like rectangles, making the errors look worse than they actually are.

[hank]

ok, I suppose I'll take that as a firm *no*.

What was the question?

- Hank

[selden]

cyber_space_doc,

There are at least two entirely different schools of thought with regard to surface textures.

One school of thought is that Celestia should be able to display the best possible planetary surfaces at the highest possible resolution. Performance is a secondary consideration. This has lead to Virtual Textures consisting of many smaller tiles. As you've noticed, if one looks at a significant fraction of a surface which is drawn using a VT, the system eventually has to load most of the VT into memory. Since VTs can be many hundreds of MegaBytes, this causes swapping and paging, both of Celestia in main memory and of the textures in graphics memory. Optimizations of the texture tiles near the poles can improve this somewhat, but it's a fundamental limitation. Keeping the in-memory footprint small would require deleting textures from memory... just to have to load them from disk again later. This just trades virtual memory I/O for higher overhead file I/O.

In other words, if you want to admire a high-resolution surface, VTs are desirable, but you have to limit what you look at if you want the performance to be reasonable. Many people want to concentrate on the complex imagery of particular regions, so that tradeoff is quite acceptable to them.

This former viewpoint is what's been expressed most vocally up to now. It's not the only one.

The other school of thought is the one that finds cubic projections desirable. It holds that planetary textures are only a backdrop for other events. Since they're usually seen only from a distance, there's no need for the overhead of VTs. When there are close flybys of surfaces, either orbital dynamics will cause them to be brief, or the observer's attention will be in other directions. Any performance degradation caused by the incidental loading of fragments of high resolution surface textures is unacceptable.

However, since the observer's attention often will be directed toward nearby bodies, it is appropriate that the quality of the surface textures be uniform. Blatant distortions like the pole-pinch effect are unacceptable. They're distracting and give the viewer a poor impression of the quality of the project. Unfortunately, pole-pinching is an unavoidable consequence of using simple cylindrical projections on 3D spherical models. The graphics hardware has to provide an increasing amount of compression of the texture near the poles. It'll be drawing texture pixels adjacent to one another which were not adjacent to one another in the original texture image. Very careful design of the texture can reduce the effects of the compression but not eliminate them.

[t00fri]

...
Optimizations of the texture tiles near the poles can improve this somewhat, but it's a fundamental limitation.
...

Do you have personal experience with the optimizations that I have incorporated into ALL my VT's??
I do not get significant overlaps of tiles ANYWHERE.

As far as I know my latitude dependent optimizations have only been published up to now for normalmap tiles via my nmtools. Of course one needs to implement analogous optimizations into the base, spec and nightlight tiles. If anyone of those is missing, the old problem is back.

Unlike you ( I suppose) I have studied the polar problematics in the greatest detail in the past.

I simply disagree with what you were elaborating above.

My entire 64k tiles setup works most smoothly near the poles, despite my ancient FX5900 card! My new tools also mount the RGB base tiles and the spec =A tiles automatically into RGBA tiles (including an alpha channel). This further contributes to reducing overhead.

Also at larger distance from the body, the lower level tiles come in, such that automatically only a few tiles are loaded.

Bye Fridger

[selden]

Fridger,

I don't doubt that your VTs operate smoothly for you. My point is about the total size: if you observe much of a VT Celestia loads much of it into memory. That can be hundreds of MB.

VTs simply aren't appropriate for circumstances where the viewpoint will only incidentally be passing closely over a large fraction of a planet's surface and where the details of the surface are irrelevant to the purpose of the flyby.

[neo albireo]

As far as I know my latitude dependent optimizations have only been published up to now for normalmap tiles via my nmtools. Of course one needs to implement analogous optimizations into the base, spec and nightlight tiles. If anyone of those is missing, the old problem is back.

Well, my Celestia pleasure certainly suffers a lot from this old problem. Zooming smoothly in on Earth is almost impossible as I always end up far too near to the ground after waiting 10 seconds texture loading. And slightly zooming out brings Earth sometimes out of sight. The problem is much worse when using the keyboard than when using the mouse. I use your normal maps and non-optimized other maps until Level 5.

I used your nmtools and was pleased with the results. However, as you already mentioned, I still lack the appropriate base, spec and nightlight tiles. As long as there is not a downloadable texture package or a simple tool package similar to nmtools allowing me to relatively easily get all optimized virtual textures, I would actually welcome cube maps.

Hardware: MacBook Pro Intel Core 2 Duo 2.33 GHz, 2 GB RAM, Bus 667 MHz, RadeonX1600 256 MB.

[t00fri]

Fridger,

I don't doubt that your VTs operate smoothly for you. My point is about the total size: if you observe much of a VT Celestia loads much of it into memory. That can be hundreds of MB.

VTs simply aren't appropriate for circumstances where the viewpoint will only incidentally be passing closely over a large fraction of a planet's surface and where the details of the surface are irrelevant to the purpose of the flyby.

That argument I even understand less.

If one watches at small FoV only ONE or at worst a few 1k x 1k tiles at level5 are loaded. Since the tile resolution has been massively reduced towards the poles there is almost NO tile overlap. Hence also near the pole I only load FEW tiles. That's not many hundreds of MB! If you work at large FoV only a few 1k x 1k tiles at level 0 or 1 are loaded. That is also not hundreds of MB. At large Fov, the operation of the VT's is entirely smooth and I get VERY high fps rates. I also see NO trace of the pinch effect at larger distances from the body that would correspond to about the closest possible zoom of a 4k-8k /one-piece/ texture.

I simply NEVER see any noticable pinches.

It is easy to verify which of my tiles are loaded.

Bye Fridger

[t00fri]

As far as I know my latitude dependent optimizations have only been published up to now for normalmap tiles via my nmtools. Of course one needs to implement analogous optimizations into the base, spec and nightlight tiles. If anyone of those is missing, the old problem is back.

Well, my Celestia pleasure certainly suffers a lot from this old problem. Zooming smoothly in on Earth is almost impossible as I always end up far too near to the ground after waiting 10 seconds texture loading. And slightly zooming out brings Earth sometimes out of sight. The problem is much worse when using the keyboard than when using the mouse. I use your normal maps and non-optimized other maps until Level 5.

I used your nmtools and was pleased with the results. However, as you already mentioned, I still lack the appropriate base, spec and nightlight tiles. As long as there is not a downloadable texture package or a simple tool package similar to nmtools allowing me to relatively easily get all optimized virtual textures, I would actually welcome cube maps.

Hardware: MacBook Pro Intel Core 2 Duo 2.33 GHz, 2 GB RAM, Bus 667 MHz, RadeonX1600 256 MB.

But that's of course no surprise: there are FOUR 64k textures overlaid in case of Earth. If only ONE of the four has the appropriately reduced polar resolution out of the FOUR, the effect of the optimization is close to NIL.

My strong suspicion is that Seldon in his extensive "explanations" above did not TEST himself FIRST how well VT's perform once all overlaid tiles are polar optimized.

My completed package for doing a full monster tile set (even with a single click) is completed since a long time. But recently, there has been a vivid development concerning high quality DXT compression (NVIDIA,... squish-based). It would be rather nonsensical, if I would release my tools package before that development has reached kind of a stationary status. I actually had to recode quite a lot of my tools, to adapt to the new situation. This means I now discard the DeVIL library altogether, produce PNG format output as default (rather than PNM), which required to implement PNG read and write methods in all my tools and to include PNG-library handling for all operating systems...etc

That sort of stuff takes it's time (which I have little of)

Bye Fridger

PS: many people also do not know about the strong correlation between the optimal tile size and their hardisk speed! If one uses too large or too small tiles from the Motherlode and combines them with the ones from my nmtools, UNNECESSARY delays from the harddisk loading will arise.

[t00fri]

As long as there is not a downloadable texture package or a simple tool package similar to nmtools allowing me to relatively easily get all optimized virtual textures, I would actually welcome cube maps.

Hardware: MacBook Pro Intel Core 2 Duo 2.33 GHz, 2 GB RAM, Bus 667 MHz, RadeonX1600 256 MB.

That argument of yours I also cannot follow.

It will certainly last longer before you might have a chance of using high-quality 8k-16k /one-piece/ cube map textures with Celestia... Not even cubemap support is yet available in Celestia, let alone high-quality cubemap textures (including spherical geometry normalmaps!) .

Either way, patience is required.

Bye Fridger