Celestia Forums

Hi all,

I think it would be usefull if we choose a standard for the tile size of virtual textures. All texture designer should try to build their virtual textures only in this format. I vote for 512 because Waltens 32k earth map is this format.

Here are the resons:
1. Celestia crash if surface map and specular map use different tile sizes and you switch through the display modes.
2. It would be more easy to build 'high' level addons for virtual texture

What do you think about this suggestion ?

Bye Jens

I am all for 512x512 myself

I agree - 512 is the way to go!

JackHiggins wrote:I agree - 512 is the way to go!

I disagree...I think the whole question is illdefined.

I made extensive tests with 32k textures of vastly different tile sizes. And I replaced my harddisk a few days ago. It was already a fast ATA100 disk (40GB) before, but now I have the fastest 120 GB EIDE hd on the market (8MB cache, 9ms access,7200rpm). It loads with 40-57 MB/s (small - large files) and is /excellent/ in burst loading. The effect on tile load performance turned out to be very significant. Of course, I benchmarked everything and got more than a factor 2 improved load performance over all.

Since people will use vastly different HD speeds and CPU's there is no simple answer to Jens' question! Moreover, the answer strongly depends on how many tile sets one will be using. My extensive tests were done both with a single 32k tileset and with 2 as well as three tilesets simulataneously (16, 32k normal map, 16 K nightlights). The performance gets dramatically worse with 2 and 3 simultaneous tilesets, if all are small, e.g. 0.5k!!

With my setup right now, I get optimal performance if I use:

0.5k base tiles (32k max)
2k normal map tiles (32k max)
2k night lights (16k)

4K non-tiled spec map.

So Jens, what would you want me to answer to your question:

Yes or no??;-)

Bye Fridger

Fridger, my problem is that the highres tiles I've built is only a addon to existing textures. If everyone use another tile size shall I build a tile set for every possible case?
Maybe other Celestia user make also high level addons which tile size will they use?

My dream is that some people in this forum find together and map the Earth with some hires tiles and all addons are compatible. Does this be so absurd?

Bye Jens

jim wrote:Fridger, my problem is that the highres tiles I've built is only a addon to existing textures. If everyone use another tile size shall I build a tile set for every possible case?
Maybe other Celestia user make also high level addons which tile size will they use?

My dream is that some people in this forum find together and map the Earth with some hires tiles and all addons are compatible. Does this be so absurd?

Bye Jens

Of course Jens, everyone agrees with your "dream" . Reality is that the optimal situalion strongly depends on hardware and the number of tile sets used.

Hopefully, Chris will find time to further tune the efficiency of the tile code such that users can hope the results will dpeend less on those mentioned degrees of freedom!

Bye Fridger

Hi all,

Fridger is absolutely right that there is a noticable performance tradeoff which will vary between systems. On the other hand, Jim is correct that we need to address the subject of standardization.

I think that for anything but earth 512x512 is probably optimal for most systems. For the earth I think 256 is a better tile size because it allows us to reduce both the install and download sizes by taking advantage of the 75% water redundancy. Mario will soon have my 256/512 version available from his website - I offer as a teaser that the install is down ~50% from the 512 version. I think there are important aesthetic reasons not to drop below 256x256 tiles. In a nutshell dxt1c compression causes small mismatches between adjacent files. This is somewhat noticable in the 512x512 tileset, but for a 128x128 tileset you would have to be blind to miss it. We could change compression format (or drop .dds altogether) but this will require a significant increase in install and download size.

Now I present a little data to back up Fridger's points. Below is a table that shows how many tiles are required on an 1152x864 resolution monitor screen for various tiles sizes. I also show in the last column the amount of graphic data which has to be loaded but is not visible on the screen (as a percentage of the screen size) - this is the inefficiency of a particular tileset. As the tilesizes get smaller they become more efficient.

Loading data off the disk drive has two major factors. The time required to find the file and the time required to pump the data to the CPU. For large tiles size the time required to send the files is dominant due to the inefficieny of the tiling scheme. For very small tile sizes the time required to find all the tiles is dominant. The seek time is roughly proportionate to the number of tiles which must be loaded and the tranfer time is proportionate to the amount of data which must be sent. To wrap up, there is a sweet spot where the two effects are about the same size. This will be different for different people's systems. For most people it will be around 512x512 size tiles.

Now to complicate things further, we need to consider the data processing overhead. This also increases in propotion to the amount of data being loaded. But it's magnitude will vary depending on the type of data (i.e. normal, base, spec, night, etc.) This is likely why Fridger finds different sweet spots for each of these different map types.

I hope this helps to provide context for understanding why it isn't easy to just pick one standard. But given the obvious advantages of a standard, this needs to be debated further.

cheers,
Walton

Bloody hell, I'm going to have another ESB!

I havent done any numbor crunching but have gone with the "feel" of how celestia loads
tiles larger than 512 seame to take to long to load and tiles smaller ,dont seam to make much of a change
And of cource i do like the .png format ,there is to much of the image lost in .dds and i just dont care for that format to much

as Denis miler says " thats just my opinion , I could be wrong"

Hi Fridger and Walton,

I understand your objections that to much tiles can reduce the performance. But I think we can reduce this effect if we map only relevant tiles. It should be possible with optimizeing of surface-,normal- specular- and nightlights map to reduce the number of tiles dramatically. From the standpoint of optimization we should go to a small tile size maybe 256.

A look at the details:
- The specular map is important at the coast line and areas with many lakes. Most parts of the oceans and some dry land areas could maped with low level tiles.
- The normal map and nightlight are only on land.
- The surface map needs on the oceans most times only low level tiles.

Ok, it's a lot work but we should do that. I'm not sure at the moment what's the best tile size for the standard 256 or 512 but we will find a solution.

Bye Jens

PS. John Van Vliet, have you tested the uncommpressed DDS formats that the latest Celestia now supports ?

jim wrote:Hi Fridger and Walton,

I understand your objections that to much tiles can reduce the performance. But I think we can reduce this effect if we map only relevant tiles. It should be possible with optimizeing of surface-,normal- specular- and nightlights map to reduce the number of tiles dramatically. From the standpoint of optimization we should go to a small tile size maybe 256.

A look at the details:
- The specular map is important at the coast line and areas with many lakes. Most parts of the oceans and some dry land areas could maped with low level tiles.
- The normal map and nightlight are only on land.
- The surface map needs on the oceans most times only low level tiles.

Ok, it's a lot work but we should do that. I'm not sure at the moment what's the best tile size for the standard 256 or 512 but we will find a solution.

Bye Jens

PS. John Van Vliet, have you tested the uncommpressed DDS formats that the latest Celestia now supports ?

Jens,

I think this is the right way to proceed!

Except that my test results seem to speek against tiles smaller than 0.5k. The HD activity increases dramatically. Could it be my slow 1Ghz CPU? In any case my HD is /very fast indeed/!!

Incidentally, where do you get these 128k textures from?

Bye Fridger

PS: I notice that since I "know" you in the forum, your English has improved immensely! Not a bad side effect I guess, but perhaps I should not notice such things since we speak the same native language;-)

Anyway, I mean it...

Hi all,

Here is another reson that speacks against to small tile sizes. The texture filtering doesn't work over the tile borders and that can be seen.

Walton wrote: For the earth I think 256 is a better tile size because it allows us to reduce both the install and download sizes by taking advantage of the 75% water redundancy. Mario will soon have my 256/512 version available from his website - I offer as a teaser that the install is down ~50% from the 512 version.

Walton, do you really think that's a good idea to build a second Earth texture? Most off us have still installed the 512 version and I see more problems then advantages with having two different VT sets. I will build my addons only for one virtual texture .

Now we found several resons for small tiles below 512 and other resons they speak for large ones. But it seems that the 512 tile size is the "golden" middle. I think we should choose the 512 for the standard or shall I say the favour tile size for texture makers.

Fridger wrote:Incidentally, where do you get these 128k textures from?

http://earthobservatory.nasa.gov/Newsro ... hp3?page=1
http://visibleearth.nasa.gov/browse.html
Of course some image processing must be done (remove clouds, deformations, color adaption) before they can be used.

Fridger wrote:PS: I notice that since I "know" you in the forum, your English has improved immensely!

Not really, I'm sitting still with my dictionary and write most post with an editor before copy this to the forum.

Bye Jens

Hi all,
This going to be just a hare off topic.
Well I guess everyone has been wondering what I have been up to. Well I am in the process of creating a 32k version of my Realistic Earth texture. It has to be done in 2 separate halves because of memory issues. I had to pull one of my 512mb sticks leaving me only a Gig of RAM because it went bad. My question is should I at this point even consider this whole project. My texture has none of the ocean redundancy that was spoken of earlier. In my texture every tile will or would be different. I am also not sure I want to have to make some 1,000+ dds tile sets for this thing. So far I would say I am 1/3 of the way done. The texture is truly beautiful and I have managed to keep the colors even better than my 16k version. But there is room for adjusting. So here is a snip of the Amazon showing the smoother transitions along the coastlines. I have also smoothed the pixels that are supposed to be water as they really shouldn’t look rough.



So what I need to know is this just a complete waste of my time. I might be interested in sharing the project if someone wants to help with what is definitely turning into a beast of a texture. Let me know what you think and if anyone wants to help me out just drop me a PM. If I don't here anything than I will assume no one else wants to try and tackle this thing and I will stick it on the back burnner or just keep it to myself.

Don. Edwards

P.S.

What do you guys use to cut the texture into nice even pieces. Is there a special plugin or a comand I am not aware of in GIMP or Photoshop?

WOW what a great looking texture Don.
I'd love to be able to use this.

But at the moment, fear my machine will not handle it too well (if at all)
I would at least have to wait untill i can afford the Terrertech 256 geforce FX card.

But my current P4-1.5 gforce 4Ti4600 128 would groan

Hi Don,

Good to see you back on the texture front .
I'm not sure if it's a good idea to build a second 32k Earth map. Sure the colors of the BlueMarble map are not the best but is it really worth to do all this work? I have also another problem. I've build some high level addons and the colors adapted to Walton's 32K Earth. This texture tiles will not correct work with your Earth.

Don. Edwards wrote:What do you guys use to cut the texture into nice even pieces. Is there a special plugin or a comand I am not aware of in GIMP or Photoshop?

I have at the moment a hand full scripts to do this job more or les automatic. See at this topic from Fridger: http://celestiaproject.net/forum/viewtopic.php?t=3089

Bye Jens

Well Jim then I will just keep this texture to myself.
The whole reason I started it was that I felt the coloring of Walton's virtual texture is just too green and with the new cloud textures I am making they do not go well together. Fridger's coloring tips were definetly incorect as far as I am conserned. So I guess all my future cloud textures will be kept to myself as well. So I guess I will just stick to keeping my textures to myself and just come in and tease everyone as time goes on. Now that you guys are locking rules on textures sizes, types, and texture coloring I guess there just isn't a place for me here any longer. So I guess I will just move on. It’s no skin off my nose as I can use my computer for things other than Celestia and it sure can free up the bandwidth I would end up wasting on further uploads. I have about five good games that I haven’t played and I can waste my time playing them just as easily as making textures.
So I guess this time I am really going to bow out. I am not angry with anyone but I feel that it’s obvious that my textures are no longer needed. I will leave my Hub sites up and do a few minor updates but I guess this is it. They will stay up as long as Chris feels its necessary and then he can close them if they take up to much bandwidth. As for any other textures that other members asked about, I think I may make a few of them available but nothing like I was planning on doing. At this point I just can’t see me wasting my time any further on projects for this forum.

DON. EDWARDS

I think that the best solution would be to have a little tool to put every part (base map, normal, spec and even night if you want) in the SAME file.
So the hard disk would only have one seek for one part of the planet.

A little tool like this is very simple to code once you have the more difficult : the tiles !

One drawback : you must have exactly the same number/size of tiles of all the textures you want to pak.

Don, I really like your earth textures! I use it as my default and I agree with your view on the blue marble colours. You can be sure that _if_ you create a 32k one I will use that one too! Now I hope that I am not the only one... By the way, it would be nice&efficient if it were possible to get your colours by batch processing the existing tiles including the hi-res extensions. I have to look into this, I believe it has been described how to do this.

Buzz,
My textures are created totally different than anyone else's I believe. My textures start out as separate layers. One layer being devoted to the oceans and water and the other is a continental layer that has the oceans and water removed. I then layer the continental texture layer on top of the water layer and proceed to blend the edges so to speak. This way by keeping the texture layers separate one can adjust the color lets say to the water without having any impact on the color of the landmasses or visa versa. This is a superior way to make textures. It does take more CPU and RAM horsepower but in the end everyone can what the results are all about. I will keep working on my texture and pop in from time to time to show how things are going. I will also experiment as to witch size tiles work best for my system and report the results to the forum for those that are interested. I just can't always take Fridger's speed tests as a given. Our machines are totally different. He has a better video card than I do a GeForceFX 5900 versus my GeForce4 TI 4600 but I have over twice the CPU power he does and my system has a dual memory channel architecture witch speeds the memory subsystem by about ten percent over most normal DDR memory based motherboards. Not to mention running at a 333HMz front side bus versus his Pentium 3 1GHz's 133MHz bus. His may be a more common design of the average computer but it doesn't give anyone with a much faster computer any idea of how fast there machine may or may not be when using these or any other textures. This is not slam on Fridger. I give him a lot of praise for trying to rein in some of the over enthusiastic users that think virtual texture are the total answer to everything. They are definitely a move in the right direction. But it is so early in the implementation of this feature that trying to lock in a certain tile size as the defacto standard, I feel is just a little premature at this time. We need more work on these super large textures so we get a real good idea of what is best for all uses. As in all things texture related in Celestia time will bare this out. It was just a few months ago we had but only two 16k texture to play with at all, the old original BlueMarble and Mars 16k dds textures. Now there are quite a few including mine and we have now moved up to virtual 32k textures. I don’t think we are going to see much beyond the 32k boundary. Number one there just are not any 64k textures floating around the web and those that do exist have to be licensed for very large amounts of money. The makers of these texture are not just going to give over there hard work for a free open source project. We might get lucky but I think we have finally reached the end of the road with respect to texture size. I also don’t feel downloading a massive amount of small mismatched tiles from all over the web and stitching them together is a good option either. This will lead to very inconsistent texture modeling. It will add a lot of the OOO-AHH factor but in reality will look rather patched together. I now what I am talking about here. Even working at the 32k level I am finding things in the textures that I have never seen before and finding that some of things we thought were snow on mountains is really clouds that were never clipped out. I am also finding other unknown objects in the textures. Now I have found some very impressive textures out there and I even found a specmap that is to die for at very high resolution. The detail is fantastic even down to the rivers and lakes of the world. But do any of us have an extra $1000 US dollars around to purchase it. Even then you can only use it for your own personal use. So we have reached a crossroads so to speak. Some of us might feel like turning around and going back the way we came. Others want to go left while others want to go right. While straight ahead is always a good option. I myself am going to pull over to the side of the road and see witch way most of the traffic is moving. I can then make a much better decision as to witch way I choose to go. Many of us will get lost on this journey and many of us may find ourselves back where we started. Its time to take a deep breath and watch the traffic and see witch way we all want to go. Just some food for thought by the way. Remember I have been in a rather philosophical state of mind lately. I think some of Rass has rubbed off on me somewhere along the line. That’s a good thing right?

DON. EDWARDS

Don I certainly hope that you do not bow out again. Your textures are simply gorgeous! I for one certainly appreciate all the work that you have done and I don't want to see you go.

Yes, I must admit that I am enjoying playing with the blue marble virtual textures becuase it is the only way that I can currently view a 32k image. But I also feel that the coloring is a little off.

Please keep up the good work, please complete your 32k image, and please release it. While I don't have a machine (yet) capable of viewing I hope to have one soon.

Thanks again for your hard work!