New concept - surface scan

[ron]

There is always a constant desire to use higher and higher resolution textures of the planets and moons so that more detail can be seen. The problem of course is that high res textures require high end graphics cards with lots of VRAM and even then you are still very limited as to how far you can go. Even looking ahead to the future it would seem that it will be a long time before we can load textures which show the level of detail that is currently availible for bodies such as the moon, earth and mars. If jpg files were assembled at the max res that is availible then they would be truely huge. Even the new earth image availble from the nasa bluemarble site shows how large they start to get. In this respect then we can safley say that you will never be able to zoom in on the moon for instance in celstia and view the surface with the sort of detail that is availible from say the clementine library.

what i propose then is that a new mode be discussed. i have not thought this through yet but i thought i would throw it on the forum to see if there was any interest. basically it goes something like this. when an object like the moon is selected in celstia there is a hotkey which will essentially center the object and then zoom in to a predefined distance above the planet in a down looking orientation. It would zoom in quite close so that only a relatively small area of the planet was visible, a bit like you would expect from an aerial or satelite photo. Then celstia would change mode to a surface scan mode for want of a better term. To do this it would need to load a new texture file, typically an EZW file. This file is not rendered on to the sphere but is loaded to a seperate viewing function. I have used viewing programs based on EZW format files before and they offer great advantages. The EZW format somehow stiches together a whole set of photos (usually air photos) into a single image file. It is not unusual for these files to be well over half a gig. The amazing thing is they load within seconds regardless of size and enable the user to pan and zoom around. All the time the program is recalculating which image tiles are visible and it also continuously adjusts the resolution so that only a small image is ever loaded in memory. Just how it does this i have no idea and the whole concept of EZW compression is mind boggling if you ever do any reading on it. But i have used it and it works great. The EZW files i have worked with however are flat. ie they represent an area of the earth as though the earth was flat and would not work for the whole planet because of it's spherical shape. What is needed is a variation on the theme so that you can pan around a globe. ie it would appear that you are panning over a flat surface but you would eventually get back to the same place. remember that you are locked in a down looking map reading style mode and there is no 3D rendering happening here. You can't look sideways or up at the stars. You can only zoom and pan the surface but there should be no limit to the detail. There may even be a way to overlay labelling like crater names etc. When you are finished browsing the surface then you toggle back to the standard celstia environment.

EZW files may not be the answer and perhaps a new format or method would work better but i think they prove the concept. This would truely make celstia a great tool for exploring the solar system.

ron

[Guest]

Actually, I think you're on the right track, but a better option may be "DjVu", a wavelet-based image compression system developed by AT&T which I think may have the characteristics we need and is available as GPL'd open source. Check it out at:

http://djvu.sourceforge.net

- Hank

----------------------------

DjVu is also an excellent format for distributing high-resolution photos, because its wavelet-based continuous tone image compression technology produces small files with very fast progressive display, seamless zooming and panning, and requires minimal memory in the client.

* DjVuPhoto (also known as IW44): a progressive compression format for color or greyscale photos, paintings, and other continuous-tone images. It is based on the mathematical theory of wavelets. File sizes are typically about 1/2 that of JPEG for a similar quality. The main advantages of DjVuPhoto over JPEG are:

* Small files: about 1/2 the size of JPEG for a similar quality. In addition, the compression ratio can be pushed up much higher than JPEG without JPEG's catastrophic artifacts.
* Progressivity: images appear very quickly on the users display, and get refined as more bits arrive and are decoded.
* Zooming and Panning with on-the-fly decompression: allows very large images to be displayed on PCs with limited memory. The image is kept in a partially decompressed, compact form in the client machine. Only the portion of it actually shown on the screen is decompressed and rendered on the fly. 4000x4000 pixel images can be displayed on machines with 32MB of RAM without disk swapping.

While this technique allows DjVu to handle large images, the maximum image size is nevertheless limited by the amount of RAM on the client machine. Another limitation of DjVuPhoto is that it supports only one internal color model (called YCrCb). These limitations may be a problem in certain applications in the geospatial imaging, medical imaging, and printing industries.


libdjvu++ 2.0: The AT&T open source DjVu reference library. It contains: a basic DjVuPhoto compressor called "c44" to convert PPM and PGM images to DjVuPhoto; a decompressor to convert DjVu 2.0 images into PBM/PGM/PPM; several utilities; and the source code of a full DjVu 2.0 decompressor, as well as a piece of the encoder (pretty much everything except the text/background segmenter and the smart and efficient DjVuText compressor).

[ron]

you may be right hank ..... it sounds promising although there still appears to be a limitation of image size based on the amount of ram. i don't think this is the case with EZW which is also a wavlet based compression method (EZW - Embedded zerotree wave). although entirely possible most users probably don't have half a gig or more of ram. in saying that though you do not need to look very far in the future to see that this will soon be the norm. luckily disk space is not the issue. even if a good image of the moon or earth requires 1gb of disk space is cheap. i support any method that will allow the planet maps to be loaded and browsed with seamless zooming and panning. mars and venus have also been imaged in very fine detail but other than downloading individual images it is difficult to explore them.

ron

[Guest]

The big advantage of DjVu is that free source code is readily available. I didn't find anything similar for EZW, and in fact I saw indications that it might have patent issues that could be a problem for using it in Celestia.

I haven't seen enough of DjVu to know for sure what it can do, but my impression is that it allows you to efficiently decompress just the part of the image that you need, to just the resolution that you need, as you need it. This would seem to be advantageous for Celestia in two ways. First, it would be possible to generate texture maps of increasing resolution "on the fly" as a planet approaches, and second, for closeup views, where only a small part of the planet is in view, it would be possible to generate texture maps that cover just the part of the planet that is currently visible. If this works, it should substantially reduce the amount of texture memory needed for high resolution closeups.

- Hank

[ron]

hank,

have a look at this site

http://www.ermapper.com/products/ecw/co ... s_body.htm

i managed to get hold off the ermapper software which supports ECW compression and it also looks good. there apperas to be open free support for this variation of the format.

ron

[Incognito]

The only problem I see with using this is that the library is windows only and the free version will only accept images for input of less than 500 megs

[Grant]

A wavelet based, projection encoded compressed file format is definitely the way to go.
Ermapper's ECW is used to hold very large (terrabyte sized) images mosaiced together from aerial and satelite sources, and the resulting files can be rapidly panned and zoomed, even across 56K modem's.
In keeping with the existing open format and source an alternative would have to be pursued, either DjVu or a varient of the recently publicised JPEG2000 wavelet method. IIRC there is an open codec available from the Uni of Adelaide Electrical Engineering Dept.

On a different note, are there plans to use the JPL ephemiris data files (DE405 etc) for precise positioning & to do the correct orbits for the terra and astra sat's etc ?

I'd be interested in assisting any implementation's.

[Grant]

Here's a funny thing - I went to the DjVu site and discovered it's originated from LizardTech who two years back went head to head with Ermapper in the U.States by bringing a court action claiming patent ownership of wavelet compression methods for geophysical / earth images.

LizardTech lost.
Now, while Ermapper keeps their precise implementation propriety, they, iirc, defended the general area for public.

I friend of friend mentioned LizardTech hawking an new improved face recognition technology post Sept 11. ahh rumours

[ron]

i recently downloaded the two (east and west) big nasa earth images mentioned elsewhere in this forum. from memory i think they totaled about 600MB as compressed tif images. i then uncompressed them which blew them out to about 6GB (don't try this at home) and stitched them together to create a single giant earth image of incredible resolution. I then used ermapper to convert the image to an ECW file which reduced it's size to about 30MB without any apparent loss in resolution that i could see. Such is the power of wavelet compression. Furthermore i can now open the image using the free ermapper viewer. It loads the entire image in a matter of 2 or 3 seconds (PIII 800 Dell 128 RAM) and allows me to seamlessly zoom and pan around the image. You can see that the resolution increases as you zoom in and only the area of the image which is visible on the screen is loaded into memory. Not only that but even this enormous image (43,200 x 21,600) does not seem to be stressing this format and it could certainly handle much larger images. if wavelet compression could somehow be incorporated into celestia then it would be possible to zoom in to extremely high resolutions.

ron

[Grant]

Ron, the Western Australian DOLA image of Perth at .4 m resolution is about 32 GB.
ECW compresses it to about 8-9GB.

Thanks for the tip, but unfortuneately I've already been wrangling ECW's on home computer for the past few years. :)

[diegorodriguez]

Yes, wavelet seems like a key technology to boast Celestia into an spectacular entertainer for all to enjoy.

But that sounds like a well defined pro's and con's competition between GPL wavelet compressors and licenced ones. Let me summarize what has been told so far:

- EZW and ECW Ermapper (defended by Ron and Grant) could do the job, but there's no GPL public licence available for them.

- DjVu (IW44) and JPEG2000 got a GPL available, but DjVu has memory limitations and licence pending issues from EZW.

Could we go a bit further in the comparisons between all those contenders by tabulating it's main advantages and drawbacks? So the developpers can deem if they have all the information needed to take a decision and if the gathered information shows a clear winner.