Just a test - 64K Moon

[t00fri]

I tried to modify it to use the Clemintine file at CM. I can't test it on Windows so................

64k-moon-texture-kit.zip

cartrite

Ah, great! I'll try to look at it under Windows XP, but probably only tomorrow...

Fridger

[danielj]

I?ll test under Windows Vista,but probably only at Tuesday...

[Chuft-Captain]

I was trying to modify it to do the moon. But I'm stuck on creating a ssc file.
Would it look like this?

Code: Select all

AltSurface "64k-Moon-VT" "Sol/Earth/Moon"
{
       Texture "64k-moon.ctx"
}


Or something like this?

Code: Select all

AltSurface "Moon:Earth II" "Sol/Earth"{
       Texture "64k-moon.ctx"
}


I don't run windows xp anymore so there is no way for me to test the script.
cartrite

Cartrite,
Here's an example of one I'm using...

Code: Select all

AltSurface "Moon16KVT (~praesepe)" "Sol/Earth/Moon"
{
   Texture "Moon16KVT.ctx"
}

The first quoted string is any name you want to appear in the right-mouse menu.
The second, is the object the texture belongs to.
So, your first version is correct. (The part where you have the name "64k-Moon-VT" can be anything you like.)

Incidentally, your second version will still work, however it will appear on the list of AltSurfaces for the Earth (with the name: "Moon"), and if selected will apply the Moon VT to the Earths surface.

CC

[cartrite]

I put this in the script

Code: Select all

Alt Surface "64k-Moon-VT" "Sol/Earth/Moon"
{
       Texture "64k-moon.ctx"
}


Incidentally, your second version will still work, however it will appear on the list of AltSurfaces for the Earth (with the name: "Moon"), and if selected will apply the Moon VT to the Earths surface.

But this is basically from solarsys.ssc that comes with Celestia.

Code: Select all

"Moon:Earth I" "Sol/Earth"

The moon still appears where it is supposed to be. Only thing different is the "AltSurface" and the extra "I". Confused me a bit. And "Yes". The moon did replace the earth. Had clouds and an atmosphere too.
I haven't actually done a 64k. Not even with Linux. Don't have the disk space.
cartrite

[t00fri]

Hello Steve,

I had a run of your script. It took 19 minutes on my 2.0 GHz Core2Duo DELL laptop. However, I think some of your scripting logistics is vasting quite a bit of time with these big files involved. I think I can reduce the required time easily by 5-7 minutes, i.e. by 30% or so.

One example:

While your initial COPY statement for the 2GB Clementine64k.bin file into level5 takes a LOT of time (~ 5 minutes! MUCH more for less powerful machines) , using MOVE instead goes instantaneously. The required tx2half commands can be easily executed also from within the leveln (n=5,4,...) subdirs by using directories in the STDOUT / STDIN redirections, such that MOVE is perfectly OK.

Fridger

[t00fri]

Steve,

just found a severe bug in your scripting: You incorrectly used 3 color channels instead of 1 (grayscale) . The file Clementine64k.bin is 8x1 bit.
The reduction to only 1 channel will also speed up the calculation a lot!

Fridger

[cartrite]

I just ran a linux version and also saw something wrong. I though that the file was color?
How many bands are in UVVIS_Clementine_Basemap_v2_ll.jp2?
I thought there were 5. When gdal_translate is run, which band gets "translated"? I didn't have enough working ram to actually open the gdal_translate file/png file so I deleted it after this morning test failed on linux.


cartrite

[cartrite]

Usage: gdal_translate [--help-general]
[-ot {Byte/Int16/UInt16/UInt32/Int32/Float32/Float64/
CInt16/CInt32/CFloat32/CFloat64}] [-strict]
[-of format] [-b band] [-expand {rgb|rgba}]
[-outsize xsize[%] ysize[%]]
[-scale [src_min src_max [dst_min dst_max]]]
[-srcwin xoff yoff xsize ysize] [-projwin ulx uly lrx lry]
[-a_srs srs_def] [-a_ullr ulx uly lrx lry] [-a_nodata value]
[-gcp pixel line easting northing [elevation]]*
[-mo "META-TAG=VALUE"]* [-quiet] [-sds]
[-co "NAME=VALUE"]*
src_dataset dst_dataset

GDAL 1.7dev, FWTools 2.3.0, released 2009/01/06

The following format drivers are configured and support output:
VRT: Virtual Raster
GTiff: GeoTIFF
NITF: National Imagery Transmission Format
HFA: Erdas Imagine Images (.img)
ELAS: ELAS
AAIGrid: Arc/Info ASCII Grid
DTED: DTED Elevation Raster
PNG: Portable Network Graphics
JPEG: JPEG JFIF
MEM: In Memory Raster
GIF: Graphics Interchange Format (.gif)
XPM: X11 PixMap Format
BMP: MS Windows Device Independent Bitmap
PCIDSK: PCIDSK Database File
PCRaster: PCRaster Raster File
ILWIS: ILWIS Raster Map
SGI: SGI Image File Format 1.0
SRTMHGT: SRTMHGT File Format
Leveller: Leveller heightfield
Terragen: Terragen heightfield
GMT: GMT NetCDF Grid Format
netCDF: Network Common Data Format
HDF4Image: HDF4 Dataset
ERS: ERMapper .ers Labelled
ECW: ERMapper Compressed Wavelets
JP2ECW: ERMapper JPEG2000
FIT: FIT Image
RMF: Raster Matrix Format
RST: Idrisi Raster A.1
INGR: Intergraph Raster
GSAG: Golden Software ASCII Grid (.grd)
GSBG: Golden Software Binary Grid (.grd)
PNM: Portable Pixmap Format (netpbm)
ENVI: ENVI .hdr Labelled
EHdr: ESRI .hdr Labelled
PAux: PCI .aux Labelled
MFF: Vexcel MFF Raster
MFF2: Vexcel MFF2 (HKV) Raster
BT: VTP .bt (Binary Terrain) 1.3 Format
IDA: Image Data and Analysis
USGSDEM: USGS Optional ASCII DEM (and CDED)
ADRG: ARC Digitized Raster Graphics
BLX: Magellan topo (.blx)

I don't think this was used in code posted here

[-of format] [-b band] [-expand {rgb|rgba}]

I know I didn't use it. It looks like it could have been in color. I think it also defaulted to translating only band 1 which was the worst in my opinion. I had a pretty good look at the 500m cube files. Every band had different null areas.
cartrite

[cartrite]

Just a thought. If you just installed the linux version of gdal and the jpeg libraries are in your library path, maybe they are causing your moon rendering problems in KDE?
cartrite

[t00fri]

I just ran a linux version and also saw something wrong. I though that the file was color?
How many bands are in UVVIS_Clementine_Basemap_v2_ll.jp2?
About that copy statement, Will "mv Clementine64k.bin level5" work in dos? It does in linux/bash.
I left the copy statement as is but I think I corrected the grayscale issue.

64k-moon-texture-kit.zip

cartrite

I am quite sure that the original 90k file is monochromatic. Here is the link to the readme.txt
ftp://pdsimage2.wr.usgs.gov/pub/pigpen/ ... readme.txt
It seems they reduced the 32bit = 3x8 bit to 8x1 bit?

Since it's lossless JP2, the size would have to be much bigger for several color channels. I used the automatic gdal_translate conversion into PNG, which is grayscale.

I have actually modified your script with MOVE. 'mv' is UNIX syntax and NOT recognized by Windows!

++++++++++++++++++++++++++++++++++++
My new version runs now MUCH faster than before.
+++++++++++++++++++++++++++++++++++

My substantially modified script now does the same thing in just 8minutes rather than 19 minutes, which is better than half the time!

Here is the modified script for Windows

Code: Select all

mkdir level5
mkdir level4
mkdir level3
mkdir level2
mkdir level1
mkdir level0
move Clementine64k.bin level5
cd level5
tx2half 1 65536 < Clementine64k.bin > ..\level4\Clementine32k.bin 
txtiles 1 65536 5  < Clementine64k.bin
del Clementine64k.bin
cd ..\level4
tx2half 1 32768 < Clementine32k.bin > ..\level3\Clementine16k.bin 
txtiles 1 32768 4 < Clementine32k.bin
del Clementine32k.bin
cd ..\level3
tx2half 1 16384 < Clementine16k.bin > ..\level2\Clementine8k.bin 
txtiles 1 16384 3 < Clementine16k.bin
del Clementine16k.bin
cd ..\level2
tx2half 1 8192 < Clementine8k.bin > ..\level1\Clementine4k.bin 
txtiles 1 8192 2 < Clementine8k.bin
del Clementine8k.bin
cd ..\level1
tx2half 1 4096 < Clementine4k.bin > ..\level0\Clementine2k.bin 
txtiles 1 4096 1 < Clementine4k.bin
del Clementine4k.bin
cd ..\level0
txtiles 1 2048 0 < Clementine2k.bin
del Clementine2k.bin
cd ..


Actually the Moon VTs look pretty impressive, if only we could find an automatic ISIS3 cleaning of the stripes and the black bars and lines.

Fridger

[t00fri]

Just a thought. If you just installed the linux version of gdal and the jpeg libraries are in your library path, maybe they are causing your moon rendering problems in KDE?
cartrite

Steve,

thanks for the suggestion, but these possibilities I have long checked.

I made sure with ldd /usr/local/bin/celestia that the linked jpeg lib is the correct one
libjpeg.so.62 => /usr/lib/libjpeg.so.62

Also I tried several moon texture formats including dds! What is most puzzling is that celestia-gtk, celestia-gnome and celestia-qt4 all work fine. So it rather looks like a KDE related library bug.

Fridger

[cartrite]

I just tried gdal_translate on the jp2 file again with "-b 2" and it did come up with an error because it was only a 1 band file.
I put the open in the script so it should work good now and have all the other folders and files needed for a VT.
I didn't think mv would work with windows but I couldn't remember what would.
These isis cube files have 5 bands though.
http://ser.sese.asu.edu/MOON/CLEM_COLOR ... est.cub.gz
http://ser.sese.asu.edu/MOON/CLEM_COLOR ... ast.cub.gz
That's what was confusing me.
cartrite

[t00fri]

I just tried gdal_translate on the jp2 file again with "-b 2" and it did come up with an error because it was only a 1 band file.
I put the open in the script so it should work good now and have all the other folders and files needed for a VT.
I didn't think mv would work with windows but I couldn't remember what would.
These isis cube files have 5 bands though.
http://ser.sese.asu.edu/MOON/CLEM_COLOR ... est.cub.gz
http://ser.sese.asu.edu/MOON/CLEM_COLOR ... ast.cub.gz
That's what was confusing me.
cartrite

Ah, good so this is cleared up, also in accord with what I felt. I knew of course that the 500m isis2 cubs were 5 band, since I extracted them individually and also found that the black bars were not in the same location in each channel...

I wasn't quite sure whether I correctly understood your final changes in your script? Does the final version differ from the one I displayed above? NOT using MOVE costs 6-7 minutes extra...

In any case I suggest you make a finalized Download post including the link to my Clementine64k.bin.zip file on CM and perhaps the latest 2.0pre2 of the F-TexTools. I will also do a link to the script and the data on the Texture Tools dept @CM.

OK?

Fridger

[cartrite]

The latest link I found to F-TexTools is version 2.0pre1
http://www.celestialmatters.org/users/t ... .0pre1.zip
I have a link to the Clementine file in a readme.txt file in the folder with the script.
Do you have a link to version pre2?
If so, I'll put that in the read me too. Then I'll make the modifications to the readme file.
The script already has the open statement. It should be almost the same as you posted. I got extra lines to change directories. That shouldn't matter. I deleted the other version that had problems.
cartrite

[t00fri]

The latest link I found to F-TexTools is version 2.0pre1
http://www.celestialmatters.org/users/t ... .0pre1.zip
I have a link to the Clementine file in a readme.txt file in the folder with the script.
Do you have a link to version pre2?
If so, I'll put that in the read me too. Then I'll make the modifications to the readme file.
The script already has the open statement. It should be almost the same as you posted. I got extra lines to change directories. That shouldn't matter. I deleted the other version that had problems.
cartrite

You are right, F-TexTools-2.0pre1.zip was the latest one I have actually published. 2.0pre2 referred to using version 2.0.6 of libNVTT for DXT compression, without allowing for any CUDA "confusion" ...But as to the tools there were no changes.

Since your 'readme's are hidden far down in the moon-vt tree I actually doubt that people would easily read the crucial info about Clementine64k.bin.zip and the latest tools link.
Why not just having them somewhere below your script link, so people just need to click and get everything needed. I am mainly worried that some old version might be used otherwise, which take significantly longer.

Oh, what do you mean with "the script has the open statement"??

Fridger

[cartrite]

So F-TexTools-2.0pre1.zip is the latest?
here is the script that is in the kit.

Code: Select all

mkdir level5
mkdir level4
mkdir level3
mkdir level2
mkdir level1
mkdir level0
move Clementine64k.bin level5
tx2half 1 65536 < level5\Clementine64k.bin > level4\Clementine32k.bin
cd level5
txtiles 1 65536  5  < Clementine64k.bin
del Clementine64k.bin
cd ..\
tx2half 1 32768 < level4\Clementine32k.bin > level3\Clementine16k.bin
cd level4
txtiles 1 32768  4 < Clementine32k.bin
del Clementine32k.bin
cd ..\
tx2half 1 16384 < level3\Clementine16k.bin > level2\Clementine8k.bin
cd level3
txtiles 1 16384  3 < Clementine16k.bin
del Clementine16k.bin
cd ..\
tx2half 1   8192 < level2\Clementine8k.bin > level1\Clementine4k.bin
cd level2
txtiles 1   8192  2 < Clementine8k.bin
del Clementine8k.bin
cd ..\
tx2half 1   4096 < level1\Clementine4k.bin > level0\Clementine2k.bin
cd level1
txtiles 1   4096  1 < Clementine4k.bin
del Clementine4k.bin
cd ..\
cd level0
txtiles 1   2048  0 < Clementine2k.bin
del Clementine2k.bin


And the readme

Download and install F-TexTools 2.0 from here. http://www.celestialmatters.org/users/t ... .0pre1.zip
Download http://www.celestialmatters.org/users/t ... 4k.bin.zip and unzip it to this folder. Be sure to save the original zip file. The working .bin files are deleted by this script to save disk space.

Then run the batch file make-moon-vt.bat. After it finishes, copy or move the moon-vt folder to your extras folder.

They have to see the readme in order to run the script. Can't be sure they will read it though.
I'll put this readme in the top directory.

Navigate to 64k-moon-texture-kit\moon-vt\textures\hires\moon.
There you will find the script and another readme. This will have further
download links. Run the bat file from that directory.
After the script finishes, copy the moon-vt folder to your extras folder

I guess maybe the script should also be in the top directory. But I can't test for typo's or paths and that could get messy. If I was only running XP
cartrite

[t00fri]

Looks OK.

Here is my 8k Clementine8k.cub file (ISIS3 not ISIS2) for practicing some (automatic) isis3 cleaning attempts.
+++++++++++++++++++
http://www.celestiaproject.net/~t00fri/images/Clementine8k.cub
+++++++++++++++++++

Would you perhaps be interested to jointly try further whether we could get rid of these stripes and black bars!? With the 8k file the cleaning routines just take seconds to end.

I have made so many different lowpass and highpass filters already and nothing helped against the stripes. Strangely enough I can perfectly clean that respective tutorial example!

Amazingly, with the GIMP destripe filter plugin, the stripes vanish instantaneously! But for >=64k GIMP is just DEAD slow...

Fridger

[cartrite]

I'm not sure what else to do with ISIS. I'll download the file and give it a try though.
This is a final moon texture kit. It should work. The readme files have been updated and I corrected another mistake in the ssc file. I'm going to delete the other one.

64k-moon-texture-kit.zip

That first file was downloaded at least 7 times. Those of you who downloaded it, you should trash it and download this one.
cartrite

[cartrite]

That 8k file will take over 2 hours to download. About 40KB/sec. How did you get a file that big onto shatters.net?
Is that the same file as the 64k version? Only smaller?
I just finished downloading the 64k file from CM. Maybe I can scale that one down?
cartrite

[cartrite]

I saved an 8k bin file from my 64k VT run on linux. I got a first look at the stripes. I thought you meant different stripes. I didn't see these with the 500 meter files I downloaded.
I imported the 8k file with raw2isis. I ran lowpass with a boxcar of 151 samples x 151 lines. I ran highpass with a boxcar of 1 sample x 151 lines. Then I used algebra to add the 2 files . This cleaned up a lot of the stripes.
I scaled this down to 4k.


cartrite