Just a test - 64K Moon

[cartrite]

After I did a run with lowpass, highpass, and algebra, I exported then imported the algebra cub file to get it back to 8 bit. Then I ran this routine with mixed results.

Code: Select all

lowpass from=Clementine64k.cub to=Clementine64k.lowpass.cub samples=701 lines=701 low=1 high=255 filter=outside
lowpass from=Clementine64k.lowpass.cub to=Clementine64k.lowpass2.cub samples=5 lines=5
handmos from=Clementine64k.lowpass2.cub mosaic=Clementine64k2.cub input=beneath

The first lowpass run seemed to only filter areas that had null pixels. The rest of the image was left alone. But this left a dark line around the former null area. After the second lowpass run the lines were still there but slightly lighter. These extra runs did "improve somewhat" some of the many null areas that were in crater shadows. A little less noticeable. Still experimenting though.

I think that isis treats all 0 values as null. When raw2isis is run, there is an option to create a min and max range for null pixels. So dn values < 5 can be set to null. I think thats how it works. I also think that the pixels with the lowest value are set to null. So if an image's lowest value dn is 3, all 3's are set to null. I'm not sure if that is intended behavior, but last year when I was working on cloud maps, I think that is what I was experiencing.
cartrite

[t00fri]

After I did a run with lowpass, highpass, and algebra, I exported then imported the algebra cub file to get it back to 8 bit. Then I ran this routine with mixed results.

Code: Select all

lowpass from=Clementine64k.cub to=Clementine64k.lowpass.cub samples=701 lines=701 low=1 high=255 filter=outside
lowpass from=Clementine64k.lowpass.cub to=Clementine64k.lowpass2.cub samples=5 lines=5
handmos from=Clementine64k.lowpass2.cub mosaic=Clementine64k2.cub input=beneath

The first lowpass run seemed to only filter areas that had null pixels. The rest of the image was left alone. But this left a dark line around the former null area. After the second lowpass run the lines were still there but slightly lighter. These extra runs did "improve somewhat" some of the many null areas that were in crater shadows. A little less noticeable. Still experimenting though.

I think that isis treats all 0 values as null. When raw2isis is run, there is an option to create a min and max range for null pixels. So dn values < 5 can be set to null. I think thats how it works. I also think that the pixels with the lowest value are set to null. So if an image's lowest value dn is 3, all 3's are set to null. I'm not sure if that is intended behavior, but last year when I was working on cloud maps, I think that is what I was experiencing.
cartrite

Meanwhile, I have also gotten rid of the black blocks that now in my cubs are bonafide NULL pixel areas. NULL=0 only in 8bit data. in 16bit NULL=-32768 and 0 is a legal pixel etc.

I used a VERY similar lowpass as you did, for filtering NULL and HRS (High Representation Saturation) pixels outside 1..220. The latter are crucial near the poles and cause that horrible pole behaviour...

I also observe that black frame around the replaced NULL pix regions. It's awkward, since the rest looks quite OK.

Also it's good to interactively apply a stretch (click that icon on the right!) and then save the result. That is easiest. You can also load the 64k stuff into qview, which doesn't take too long even. So you may directly check the amount of stretch in the big 64k file.

The result will not look perfect, but A LOT better than the uncleaned 64k Celementine input.

Fridger

[cartrite]

I used a VERY similar lowpass as you did, for filtering NULL and HRS (High Representation Saturation) pixels outside 1..254. The latter are crucial near the poles and cause that horrible pole behaviour...

I'll give that 254 a try. With 255, the polar areas were white and the former null areas were changed to a light gray.
I'm wondering if the first extra lowpass run should have a much bigger boxcar, say 1501x1501 or even larger.
cartrite

[t00fri]

I used a VERY similar lowpass as you did, for filtering NULL and HRS (High Representation Saturation) pixels outside 1..254. The latter are crucial near the poles and cause that horrible pole behaviour...

I'll give that 254 a try. With 255, the polar areas were white and the former null areas were changed to a light gray.
I'm wondering if the first extra lowpass run should have a much bigger boxcar, say 1501x1501 or even larger.
cartrite

I would guess so. You must scale the boxcar up to the appropriate dimensions, since boxcars are supposed to encompass the structures that are going to be affected/filtered

Fridger
Actually I used 220 for the HIGH boxcar limit(254 was a typo)

also I used the percentage syntax for MINOPT or so.

Fridger

[danielj]

Someone can provide a smaller base texture,i.e,4 or 8k to me to test?

[cartrite]

I've been doing all this latest work on the 64k (2gb) cub. The initial stripes are gone. The VT looks a lot better. But I'm still not satisfied with all the leftover null areas. Can't figure out why that dark stripe surrounds all those bigger null blocks. Maybe I'll stumble on to something soon.
cartrite

[t00fri]

Someone can provide a smaller base texture,i.e,4 or 8k to me to test?

Daniel,

due to the large up-download speed asymmetry, you can download the big Clementine64k.bin.zip file at about the same rate as it takes me to upload Celementine8k.bin.zip.

Running cartrite's script takes only 8 minutes on the 64k file!!
So what do you want? With 4k or 8k you would have to modify all names in the script and surely would enter a typo...

We are presently working hard to try and clean a number of artefacts in the big 64k file.
Since the file is so big, it's not quite simple. So you don't need to complain about black bars and stripes. We DO know about it. It's not our fault . We will get rid of them soon.
These artefacts are NOT related to my F-TexTools, but contained in the scientific input data. There are NO really good imaging data of the Moon unfortunately. Still 64k is VERY detailed and hence quite worthwhile...

If you once know how to do things, it will only take 8 more minutes to do it once more...


Fridger

[danielj]

I have another problem.I NEED TO USE THE ADMINISTRATOR ACCOUNT to change the solarsys.ssc.How can I do this?I have 3 separate Celestia installations and I HAVE to put some of them inside Program Files

[cartrite]

I didn't have much luck getting any better results with the null areas. So far, I tried lowpass with may different options, fillgap (took too long), and my last attempt, I tried kernfilter with the file "gaussianSquare5x5.txt". This program will take a looooooong time to run but may be the best choice. I ran it for about a half an hour and only 20% was completed. But I was able to view the unfinished cub in qview and see that the results were not too good.

Maybe if a kernel file was created with a bigger boxcar"gaussianSquare25x25.txt"? Or an irregular shaped kernel with a bigger boxcar.

The gimp could probably do it with selecting the null areas, expand the selection by an unknown number of pixels, and do a gaussian blur on the selection but I think it would take forever on a file that big.
cartrite

[t00fri]

Steve,

I am also a bit frustrated, after critically inspecting the resulting quality of my NULL-pixel cleaned 64k Moon texture. In summary, the Clementine data are simply not sufficiently good to warrant such "time expensive" efforts. Anyway, it was certainly useful to gain further practice with all these ISIS3 filter routines etc.

Fridger

[cartrite]

JAXA Kaguya
Maybe they will release a global map of the moon soon. I know they must have taken enough images by now.
In any case, LRO will be up there in a couple months.
cartrite

[danielj]

I actualy have a 32k VT map of the Moon by John van vliet.How is this map compared to Clementine?s 64K?Is it too crude or imprecise?Maybe a little ficctional?

[John Van Vliet]

--- edit ---

[bdm]

Is it possible to make VT textures with your tools using one of my own textures? Like if i made 16k planet texture or something?

In the Pictures thread, I posted a picture of a planet texture that I made here. The texture itself is nothing special, but it is a 16K VT that I made with the good Doctor's VT tools, and you can see that the closeup is quite detailed.

I didn't want to take the time to learn all the intricacies of the tools, so I did what anyone sensible would have done - set it up once, experiment until I got it working, then turn the result into a BAT file.

Here is the batch file that I use to split up a 16K texture. The batch files assumes that the VT commands are in the path and that the subdirectories do not exist. The batch file takes one parameter, the name of the file that is to be split into a VT.

Code: Select all

@echo off
REM ---
REM This batch file assumes an 16k x 8k PNG map
REM ---
if "%1."=="." goto error
mkdir level0
mkdir level1
mkdir level2
mkdir level3
mkdir level4
cd level0
png2bin.exe < ..\%1 | tx2half.exe 3 16384 | tx2half.exe 3 8192 | tx2half.exe 3 4096 | tx2half.exe 3 2048 | txtiles.exe 3 1024 0 9
cd ..\level1
png2bin.exe < ..\%1 | tx2half.exe 3 16384 | tx2half.exe 3 8192 | tx2half.exe 3 4096 | txtiles.exe 3 2048 1 9
cd ..\level2
png2bin.exe < ..\%1 | tx2half.exe 3 16384 | tx2half.exe 3 8192 | txtiles.exe 3 4096 2 9
cd ..\level3
png2bin.exe < ..\%1 | tx2half.exe 3 16384 | txtiles.exe 3 8192 3 9
cd ..\level4
png2bin.exe < ..\%1 | txtiles.exe 3 16384 4 9
cd ..
goto end
:error
echo Must supply filename
:end


After running this script, create a subdirectory in the textures/hires folder, move the level* directories there, and create in the textures/hires folder a file with a .ctx extension with contents like this (substitute <name of subdirectory> with the name of the subdirectory):

Code: Select all

VirtualTexture
{
    ImageDirectory "<name of subdirectory>"
    BaseSplit 0
    TileSize 512
    TileType "png"
}


These texture tools are worth taking the time to learn. Even if all you do is make a batch file and just run it every now and again, the results are worth it.

[BobHegwood]

I didn't want to take the time to learn all the intricacies of the tools, so I did what anyone sensible would have done - set it up once, experiment until I got it working, then turn the result into a BAT file.

Now that is something which should be quite useful for people who are new to the VT world.
Thanks very much for this idea (and the code, of course.)

Take care, Brain-Dead

[t00fri]

I didn't want to take the time to learn all the intricacies of the tools, so I did what anyone sensible would have done - set it up once, experiment until I got it working, then turn the result into a BAT file.

Now that is something which should be quite useful for people who are new to the VT world.
Thanks very much for this idea (and the code, of course.)

Take care, Brain-Dead

While I am always VERY happy to see more scripts for the F-TexTools coming in, let me emphasize that bdm's script would not be directly applicable to the Moon texture, since that only involves 1 color channel (grayscale image) instead of 3 (normal RGB color image) as assumed in bdm's script.

For the Moon there is the script by cartrite (aka Steve) in this thread that I have also tuned a bit in collaboration with him.

Fridger

[BobHegwood]

While I am always VERY happy to see more scripts for the F-TexTools coming in, let me emphasize that bdm's script would not be directly applicable to the Moon texture, since that only involves 1 color channel (grayscale image) instead of 3 (normal RGB color image) as assumed in bdm's script.

For the Moon there is the script by cartrite (aka Steve) in this thread that I have also tuned a bit in collaboration with him.

Fridger

All are appreciated by the Brain-Dead...

[bdm]

While I am always VERY happy to see more scripts for the F-TexTools coming in, let me emphasize that bdm's script would not be directly applicable to the Moon texture, since that only involves 1 color channel (grayscale image) instead of 3 (normal RGB color image) as assumed in bdm's script.

Indeed. I just pasted the script and forgot to mention the number of color channels. To make my scripts work with 1 color channel, change the 3 to 1 wherever it is found:

tx2half.exe 3 16384
change to
tx2half.exe 1 16384

If you are not sure how many channels there are in your image, cut the script down so it just does the level 0 image, run it, and then inspect the two tiles. If the number of color channels is set correctly, the two halves should look like the left and right side of the image. If it is not right, you will see some odd effects such as twin images and odd color effects.

[t00fri]

But I'm still not satisfied with all the leftover null areas. Can't figure out why that dark stripe surrounds all those bigger null blocks. Maybe I'll stumble on to something soon.
cartrite

I solved that problem after some further thinking about what's going on at the NULL pixel front

The solution sounds easy enough:

Firstly, we agree there is a way to replace ALL NULL pixels by the boxcar average, while leaving the other normal pixels unaffected. This also worked OK for both of us, but as a result, there always remained a dark narrow frame arround the previous NULL pixel area in the end. Since this was quite a conspicuous artefact, it was unacceptable...

The key point is that these dark pixels were NOT NULL pixels and hence they were not affected in the above operation. This you can study clearly by coloring the special pixels with the respective tool in qview!
Here is an illustration. Only the blue pixels are NULL pixels! The dark frame pixels are apparent:


+++++++++++++++++
All you got to do before elimination of the NULL pixels with the lowpass filter, is to convert the dark frame pixels into NULL pixels
+++++++++++++++++

This I did like so:

Code: Select all

# replace dark 1-3 pixel frames around NULL pixel areas by NULL pixels, too

noisefilter from=Clementine64k.cub to=Clementine64k_nullpix.cub tolmin=15 tolmax=255 samples=3 lines=1 replace=null

noisefilter from=Clementine64k_nullpix.cub to=Clementine64k_nullpix2.cub tolmin=15 tolmax=255 samples=1 lines=3 replace=null


This only associates the pixels of the narrow frame, satisfying TOLMIN=15, as noise
The action for noise is replace=null.
Remember the meaning of TOLMIN, TOLMAX?

Note the second lowpass has samples and lines of the boxcar interchanged. It typically converts the horizontal frame pixels to NULLs, while the command above it does the same job for the vertical frame pixels.

The rest remains as before. I now have a 64k Clementine64k.bin input texture that via
your script generates VTs where the previous NULL areas have become VERY unconspicuous. They can only be found after intensive searching in high resolution mode!

Here is the cross check with my final 64k VT set made from that input. After a little searching I am sure you will find the triangular, formerly blue NULL pixel area in this Celestia screenshot: The reduced resolution is of course unavoidable, since all null pixels have been replaced by the AVERAGES of the sourrounding pixels...



Fridger

[cartrite]

That looks pretty nice. How does it do with the small irregular null areas in the craters around the poles? I was getting gray fill and surrounding area was almost white.
cartrite