Specular maps and Normalmaps together

[cartrite]

This problem goes back to version 1.5.1. If I use a normalmap on earth with a specularmap the color gets all messed up. As far as I can tell, sometime after version 1.5.1 and before 1.6.1 a feature was added to normalmaps to use specular highlights. This allowed spec maps for a normalmap. But it conflicts with the specmap used for highlighting water on the earth texture. The only way I could find to deal with it was lowering the specular color. But that makes the spec map almost unusable.

This image shows a normalmap and specmap together running on version 1.5.1.

This image shows a normal and specmap running on version 1.6.1 and looks the same up to the current version.

I've looked though the code for the past couple of days and couldn't find what was done to cause this so ........
I think it probably has something to do with adding specular highlights on normalmaps. I'm not aware of any workaround.

cartrite

[John Van Vliet]

without even knowing what your hardware is i can not even guess
what is your 3d card and WHAT driver are you using - also WHAT is the Operating system ?
Linux
Apple
MS Windows XP,7,8,10, what ?

i have no such issue and have not had this issue in all my years


the only thing i can GUESS is your hardware is not using openGL or the Nvidia drivers

[cartrite]

The hardware list is in my signature. OS is Tumbleweed. I took the first image using the windows version 1.5.1 running on wine. That one looks right. The other was running the gtk version 1.6.1. But it looks same on qt5 running 1.7.0 beta. Current source. Open GL 3.0. If nobody else has this problem it's probably a driver issue.

[onetwothree]

.

Hi, could you provide cel url and changed celestia data files if any so we can check on different hardware?

[cartrite]

Code: Select all

cel://Follow/Sol:Earth/2019-03-17T20:52:37.95569?x=cHKSJfVkBAE&y=L9KNcHYx&z=WrEuiGiyMQ&ow=0.77054&ox=0.00189109&oy=-0.637387&oz=0.0017144&select=Sol:Earth&fov=29.8169&ts=1&ltd=0&p=0&rf=4164738951&lm=2048&tsrc=0&ver=4

I just got 1.6.1 gtk running normally. It depends on the shaders folder being in the main directory. /usr/local/share/celestia. Running from a terminal, it complains if there is no shader folder and gets an error loading ARB vertex program.

Code: Select all

Initializing ARB vertex programs . . .
Loading ARB vertex program: shaders/diffuse_arb.vp
Error loading ARB vertex program: shaders/diffuse_arb.vp
render path: 8

With the folder present, it loads it and all looks good. 1.6.1 still has the other render modes but it is running in opengl2.0.

Code: Select all

Initializing ARB vertex programs . . .
Loading ARB vertex program: shaders/diffuse_arb.vp
Loading ARB vertex program: shaders/specular_arb.vp
Loading ARB vertex program: shaders/haze_arb.vp
Loading ARB vertex program: shaders/bumpdiffuse_arb.vp
Loading ARB vertex program: shaders/bumphaze_arb.vp
Loading ARB vertex program: shaders/shadowtex_arb.vp
Loading ARB vertex program: shaders/diffuse_texoff_arb.vp
Loading ARB vertex program: shaders/rings_arb.vp
Loading ARB vertex program: shaders/ringshadow_arb.vp
Loading ARB vertex program: shaders/night_arb.vp
Loading ARB vertex program: shaders/glossmap_arb.vp
Loading ARB vertex program: shaders/diffuse2_arb.vp
Loading ARB vertex program: shaders/haze2_arb.vp
Loading ARB vertex program: shaders/diffuse_texoff2_arb.vp
Loading ARB vertex program: shaders/specular2_arb.vp
Loading ARB vertex program: shaders/night2_arb.vp
Loading ARB vertex program: shaders/star_arb.vp
Loading ARB vertex program: shaders/multishadow_arb.vp
Loading ARB vertex program: shaders/texphong_arb.vp
Loading ARB vertex program: shaders/texphong_alpha_arb.vp
Loading ARB vertex program: shaders/ell_galaxy_arb.vp
All ARB vertex programs loaded successfully.
render path: 8


When I look at the shaders log in 1.7 it seems to have the programs there but....... The 1.7.0 beta versions don't read from the shaders folder and they look the same. Version 1.6.1 is reading / loading files from /usr/local/share/celestia. The data files, solarsys.ssc and celestia.cfg do this out of the box. No changes. I did play with SpecularColor and it seems to color the ocean parts of earth with whatever color is present in the SpecularColor value in solarsys.ssc. The normal map was made with nmtools-2.0pre2. Tried both png and dxt5nm formats. Does it with BumpMaps too. The spec map is in the alpha channel but I did a rgb texture of earth and did a separate spec map. Didn't work.

cartrite

[cartrite]

Celestia-f48a72f9bc97d7e17da6ecd3952d0f9c9af36ed0

This builds and runs fine. Committed on Feb. 24 2018.Soon after this, and I'm not sure when exactly, normalmaps act strangely on my system. Gonna try and see what is the real cause but a lot has changed since then.

[onetwothree]

Let git bisect help you

[cartrite]

Celestia-e290322b1c04cbbc75c9c1f354a7081dd8dd57a3

That was the last commit that worked normally. The next commit described as Finish eigenization

Celestia-85829063e0683e6de4c824f9f26121bc36f21536

was broken. Not what I was seeing above. It showed no earth. No png or dds files were loading. I'm not sure when they started being visible again. Looking..........

Added after 1 hour 24 minutes:
Somewhere between commits 73a869c and f478faa earth becomes visible again with too much specular.
Celestia-f478faa5a419f14aae4968a40db334468fbfee8b

Celestia-85829063e0683e6de4c824f9f26121bc36f21536

Celestia-e290322b1c04cbbc75c9c1f354a7081dd8dd57a3

[onetwothree]

Hi cartrite,

Thank you for investigations!

If you have time, could you test 85829063 with 8a894bfb cherry-picked? And then with 4ad277ed. (85829063 has an error in frustum computation and 8a894bfb provides an workaround for it (i still fill shame for publishing such terrible code) while 4ad277ed has a proper fix.)

Could you also send me files you used to reveal the error? So that I can reproduce the error.

[cartrite]

OK, 8a894bfb fixed the problem, or worked around the problem completely. Looked normal. But 4ad277ed broke it again. The planet was visible but too much specular like above.

[onetwothree]

Hmm, i will try reproduce it locally. Basically 8a894bfb and 4ad277ed do the same thing (compute inverted planet view matrix), but 4ad277ed does it properly from vector math's point of view, i bet they produce they same results.

[cartrite]

I'm sure they do. But something else may have been changed between the 2 commits. When I was trying current code, I was changing specular color in solarsys.ssc. The color of the oceans were the color I used. So somehow, by using a normalmap, the specular map was ignored and only the specular color was used?

[onetwothree]

So, you misunderstood me. My idea was that you use 85829063 as a base, then apply 8a894bfb and 4ad277ed only (using git cherry-pick). But nevermind, with your dataset I can reproduce the issue so I will do this myself.

The only possible change between 8a894bfb and 4ad277ed is vertex program removal, but in GL2 mode they were used for stars only. I need to check this better.

[cartrite]

F478faa5 was just before 4ad277ed. The captured image from f478faa5 was posted above. 4ad277ed looked the same. In fact I downloaded 4ad277ed last night and was gonna try it this morning next. I was unaware that 8a89 had fixed the invisible planet. That seemed to be the last commit that rendered the planet correctly on my system. So my real problem was caused between that commit and 4ad277ed.

Added after 5 minutes 1 second:
I'm not too good at using git. I was unaware that cherry pick was a command. I couldn't even figure out how to use bisect. I just went on github and downloaded zips quite a challenge.

[onetwothree]

cartrite, are you sure that your e290322 is correct? With files you sent I have the following:

e290322:

Spoiler

the same for 8a894bfb:

Spoiler

f478faa5 and 4ad277ed look differently:

Spoiler

I even compiled old commit a1043fb, it looks like e290322:

Spoiler

[cartrite]

Although its possible that commit b9d532a90 caused the problem, I checked the next one

Celestia-6653929310fc6a6bcf7bb3151285031cd93b4252

and the problem was there. So maybe something was forgotten when removing vertexprog.cpp from the library or there was a bool statement in render.cpp removed that related to bumpMapping. Or maybe it was disabling done in the prior commit.

Added after 1 hour 10 minutes:
Yeah. Im sure. That long number was copied from the folder. I downloaded the zip file from github. Then extracted it. It put the name on the extraction folder.

I'm using libpng16. That's what created the textures. I think it did. I know the normalmap was cause I had to build nmtools. But the bmng files may have used other. I created that texture with f-tex-tools and they were probably built with a different png lib. Not sure if that would matter though. Strange that you get no good results. When I get back, I'll try to post the zip file for the textures here and maybe someone else can try.

Added after 1 hour 32 minutes:
If anyone else want to try and test this, this is a virtual texture created by FTex-Tools-2.0-pre2 and nmtools-2.0pre2. It's levels 0 and 1 . All the folders and ctx files included. It contains the July version of the Blue Marble (world.200407.3x86400x43200.bin )with a specular map in the alpha channel. It also has a normalmap that used the srtm_ramp2.world.86400x43200.bin file. There is also a data folder with solarsys.ssc. If you have a working version of Celestia 1.7.0 it will probably show problems as above.

Celestia_QT.zip

Added after 24 minutes 8 seconds:
b9d532a is were I start getting the problem. 8a894bf and 717bf34 both work correctly.

[cartrite]

I noticed something strange today. I was getting similar results that you were getting. All the commits I thought were working were showing results that you were getting.
This was after I renamed the shader folder to stop its use. To make a long story short, You seem to need the shader folder in Celestia_QT folder or your top celestia working folder. For these versions, to get results I was getting, the shader folder must be present. At least on my system.

[onetwothree]

cartrite, could you confirm that https://github.com/CelestiaProject/Celestia/tree/fix-specular works fine?

[cartrite]

Thank You Much. Fixed!!!!!!!!!!!!! I had a feeling it had something to do with lodspheremesh.cpp. Had no idea how to fix it though.

[onetwothree]

That's was an example of a really bad code. A hidden, implicit dependency between VP and GLSL code paths. But thanks to your help it was quite easy to fix it.