Problem with Virtual Textures

[bdm]

I have managed to follow Bob's "Virtual Textures for Dummies" guide and have now got virtual textures working, except for one problem.

Some of the texture tiles are exporting with a nasty black line through the middle of them.

These black lines are fairly reproducible; they appear at the same level for each level, although different levels of VT have the line in a different place. Thus, zooming in and out of the planet cause the black line to disappear (and reappear elsewhere if the new level is also affected by the issue).

Has anyone encountered this problem before? What caused it? What is the best way to fix it?

[t00fri]

If you want sensible feedback the first info required would be WHOSE VT's you are talking about!? You would have certainly avoided these artefacts had you done your VT's yourself with the help of my F-TexTools (using txtiles and png2bin).

The occurence of such black lines is not uncommon, since a number of people who have fabricated such VT's (-> ML) don't seem to know about certain traps... I can come up with a number of reasons for such black lines, depending on the kind of texture and the manipulations performed with them.

One frequent reason is "sharpening" applied in form of "Unsharp masking" or related techniques. Just read out the respective texture width. Is one pixel missing or is the last pixel column black? Software bugs are another possible reason. This often happens when people take recourse to non-mainstream image manipulation software...

F.

[BobHegwood]

If I may...

This problem can also come about as the result of corrupt data imbedded in the parent images from which your VT's are being created. Please make certain that you are using error-free sources for your VT's.

Thanks, Brain-Dead

[Reiko]

If you want sensible feedback the first info required would be WHOSE VT's you are talking about!? You would have certainly avoided these artefacts had you done your VT's yourself with the help of my F-TexTools (using txtiles and png2bin).

The occurence of such black lines is not uncommon, since a number of people who have fabricated such VT's (-> ML) don't seem to know about certain traps... I can come up with a number of reasons for such black lines, depending on the kind of texture and the manipulations performed with them.

One frequent reason is "sharpening" applied in form of "Unsharp masking" or related techniques. Just read out the respective texture width. Is one pixel missing or is the last pixel column black? Software bugs are another possible reason. This often happens when people take recourse to non-mainstream image manipulation software...

F.

Is it possible to make VT's with just those tools and avoid going through the cygwin, netpbm, imagemagick hair pulling nightmare?

[BobHegwood]

Is it possible to make VT's with just those tools and avoid going through the cygwin, netpbm, imagemagick hair pulling nightmare?

Reiko,

Have a look at Dr. Schrempp's new tools HERE.

[Reiko]

Is it possible to make VT's with just those tools and avoid going through the cygwin, netpbm, imagemagick hair pulling nightmare?

Reiko,

Have a look at Dr. Schrempp's new tools HERE.

More command line tools to give me a nose bleed trying to figure out. lol

Thank you though.

[BobHegwood]

More command line tools to give me a nose bleed trying to figure out. lol

Thank you though.

Yeah, I hear you.

This is as good a time as any to ask Selden though...

Does my very OLD and inadequate explanation for VT's need to be a sticky, Selden?
Mayhap we would be better served by removing that thread entirely? Just a thought.

Thanks, Brain-Dead

[t00fri]

If you want sensible feedback the first info required would be WHOSE VT's you are talking about!? You would have certainly avoided these artefacts had you done your VT's yourself with the help of my F-TexTools (using txtiles and png2bin).

The occurence of such black lines is not uncommon, since a number of people who have fabricated such VT's (-> ML) don't seem to know about certain traps... I can come up with a number of reasons for such black lines, depending on the kind of texture and the manipulations performed with them.

One frequent reason is "sharpening" applied in form of "Unsharp masking" or related techniques. Just read out the respective texture width. Is one pixel missing or is the last pixel column black? Software bugs are another possible reason. This often happens when people take recourse to non-mainstream image manipulation software...

F.

Is it possible to make VT's with just those tools and avoid going through the cygwin, netpbm, imagemagick hair pulling nightmare?

Of course, that's why I wrote the tools. You need NOTHING else but the tools and the Windows command line console. Of course you need to download the scientific raw data, which are pretty big files. So you need a reasonably fast line and enough HD space.

f.

[selden]

Does my very OLD and inadequate explanation for VT's need to be a sticky, Selden?
Mayhap we would be better served by removing that thread entirely?

I think that's up to you.
You should be able to edit its initial entry to say something appropriate. I noticed that its username seems to be spelled somewhat differently than the one you're using now (a space between Bob and Hegwood). I think phpbb v3 will let me change the owner of the initial post, but I think it would be better to give that old username a new password if you've forgotten it.

(I haven't done anything with VTs for a very long time, and am concentrating on other aspects of Celestia at the moment, so I'm a poor choice to make comments on the best ways to create and use them.)

[bdm]

If you want sensible feedback the first info required would be WHOSE VT's you are talking about!? You would have certainly avoided these artefacts had you done your VT's yourself with the help of my F-TexTools (using txtiles and png2bin).

The occurence of such black lines is not uncommon, since a number of people who have fabricated such VT's (-> ML) don't seem to know about certain traps... I can come up with a number of reasons for such black lines, depending on the kind of texture and the manipulations performed with them.

One frequent reason is "sharpening" applied in form of "Unsharp masking" or related techniques. Just read out the respective texture width. Is one pixel missing or is the last pixel column black? Software bugs are another possible reason. This often happens when people take recourse to non-mainstream image manipulation software...

I created the texture myself using Fractal Terrains, Blue Marble 2 and Photoshop 7. I used Photoshop to resize the images directly from the PSD-format master file which is 8192x4096. The resized images were saved in PNG format and had dimensions that were powers of 2. These images had no black lines. I used the virtualtex script running with cygwin and ImageMagick to create the virtual textures.

I have found three black lines:
* one in the bottom row of the level 1 images (tx_0_1 to tx_3_1) slightly below halfway - shown in my previous post above.
* one in the top row of the level 2 images (tx_0_0 to tx_7_0) about 3/4 of the way down the tile (see below).
* one in the top row of the level 3 images (tx_0_0 to tx_15_0) slightly above halfway (see below).

The lowest level tiles had no black lines.

These black lines seem to appear in roughly the same place when the images are exported as JPEG files instead of PNGs. I also get similar black lines when I create virtual texture files using the 8kMercury.png that I downloaded from the Motherlode a while ago. This strongly suggests that the issue is independent of which 8K image file I use as the master.

At this stage I believe that the most likely cause of the issue is a bug in ImageMagick when running on my system but I cannot be sure.

I am primarily doing this to learn more about virtual textures. I will try your new F-TexTools as well when I get a chance to download them.

[t00fri]

If you used my virtualtex script, then I most probably know the reason of these lines. Virtualtex strongly depends on Imagemagic to do the tiles. Unfortunately new Imagemagic versions are VERY frequent and often badly tested. IM had this problem with generating extra black lines within a range of versions that I don't remember anymore precisely.

If you cut out the tiles with my new txtiles from the F-TexTools, then this is MY code which is very well tested and does not have such artefacts. Also txtiles works MUUUUUUUUUUUUCH faster than virtualtex.

F.

[bdm]

Yep, I thought that Imagemagick was the problem. To me, the test with the Mercury texture was the clincher.

I'm going to install your new F-TexTools tools tonight and try them out.

[t00fri]

Yep, I thought that Imagemagick was the problem. To me, the test with the Mercury texture was the clincher.

I'm going to install your new F-TexTools tools tonight and try them out.

That's certainly the best way. Note, since txtiles is made originally for handling scientific "monster" input textures in 1x8bit, 3x8bit or 4x8bit binary format, you will first need your ficticious RGB texture in PNG format and convert it to 3x8 bit binary with my png2bin command. Note also that all my tools support the elegant pipe mechanism. This is explained with some examples in my nmtools tutorial. The output format of txtiles is again PNG format. You may later convert it with many other tools into JPG or DDS.

http://www.celestialmatters.org/?page_id=39

The F-TexTools work in a very similar manner than the nmtools. The latter are for making normal maps.

F.

[bdm]

I have downloaded the F-TexTools from this location:
http://www.celestialmatters.org/users/t ... e4.bin.exe

When I run it, I get the error message: "Program too big to fit in memory".

[CAP-Team]

What Windows version are you using, and how much RAM does your machine have?

I've used the tools with no problems on Windows XP with 2 GB of RAM.

[John Van Vliet]

edit 5:46 pm

[bdm]

I have Windows XP with 1GB of RAM.

[t00fri]

I have Windows XP with 1GB of RAM.

THAT is NOT much, of course... given that Windows alone takes > 300 MB for operation... The size of the Win installer pack that gave you problems is 430 KB only!

Well we never had this complaint before, and honestly, I don't understand it, since my utilities are really not big programs. Perhaps you had some other applications running that took all your memory? How about a fresh restart and trying it once more? Was Celestia running, perhaps?

You can always try the non-installer (zip) version. It also has the Windows binaries and libs in it's Win32_PC.bin subdirectory.
You then should add the PATH by hand into the Windows environment dialog as described in detail in my nmtools tutorial.

F.

[t00fri]

input is .png and output is .png and takes about 8 min to cut a 16384x8192.png map

John,

my God, you don't know what you are missing [despite being the author of the good old virtualtex script ... ]:

On my OLD single core machine, I timed the tiling of a 16k Mars RGB texture in PNG format to give 128 polar-optimized PNG output tiles:

It's this VERY simple single statement for getting level3 1k x 1k tiles.

Code: Select all

png2bin < mola16k.png | txtiles 3 16384 3


Here is the timing setup:

Code: Select all

time bash -c 'png2bin< mola16k.png| txtiles 3 16384 3'
[txtiles]: Input file is a 3x8 bit RGB color map: 16384 x  8192

           Generating  128 optimized VT tiles for level 3
           in PNG format,of size from 512 x 1024 to 1024 x 1024

           PNG_compression level = 1 (range: 0..9)

[png2bin]: Converting PNG format to binary output
           bpp: 3  size:  16384 x  8192
bin [  1024 rows of  8192 ->   1.33 s]
tile[    16 VT's of   128 ->   3.91 s]
bin [  2048 rows of  8192 ->   2.65 s]
tile[    32 VT's of   128 ->   9.97 s]
bin [  3072 rows of  8192 ->   3.88 s]
tile[    48 VT's of   128 ->  16.33 s]
bin [  4096 rows of  8192 ->   5.15 s]
tile[    64 VT's of   128 ->  22.74 s]
bin [  5120 rows of  8192 ->   6.91 s]
tile[    80 VT's of   128 ->  29.43 s]
bin [  6144 rows of  8192 ->   8.73 s]
tile[    96 VT's of   128 ->  36.02 s]
bin [  7168 rows of  8192 ->  10.43 s]
tile[   112 VT's of   128 ->  42.61 s]
bin [  8192 rows of  8192 ->  11.81 s]
tile[   128 VT's of   128 ->  45.96 s]
bash -c 'png2bin< mola16k.png| txtiles 3 16384 3'  55.41s user 2.39s system 99% cpu 58.336 total


++++++++++++++++++++++++++
So with the F-TexTools a 16384x8192 .png tiling job is done on an old machine in

55 seconds!
++++++++++++++++++++++++++

NO other packages needed etc. The F-TexTools package contains the sources AND binaries for Linux, Windows and MacOS. The Win 32 binaries are NATIVE code not CYGWIN. An installation of CYGWIN is not required. Just use the Windows command shell ('cmd').

F.

[cartrite]

More command line tools to give me a nose bleed trying to figure out. lol

Thank you though.

After installing F-Tex-tools, You can have a look at the 64k-bmng-texture-kit.zip found here.
This will create a 64k texture with a couple of mouse clicks. You can also look at the scripts to get a general idea of what is going on and modify them to suit your needs.
cartrite