Another preview - cloud shadows

[Jeam Tag]

Another screenshot, this time using Don Edwards's 4k cloud texture

Very impressive, Chris! What a great feature. Jeam

[Chuft-Captain]

Firstly I'd just like to say this work and your other work on improved rendering looks absolutely awesome Chris. Unfortunately I won't be able to benefit from any of this as I don't have OpenGL 2.0 graphics.
This could well be the release that finally forces me to upgrade my desktop memory and graphics (perhaps to a Nvidia card).

Celestia will continue to support older graphics hardware, but a lot of the new rendering methods are only practical on OpenGL 2.0/DX9 capable hardware.

Yeah, realize that. I knew that you wouldn't want to force people to upgrade. (When I said "forces me to upgrade", what I really meant was "motivates me to upgrade".

Just one question about the following list of priorities. I see that depth-sorting issues is not on the list. Have you given up on that Chris?, or just forgot to mention it? Or have you fixed it already? (***typed with a hopeful tone*** )

I should have put that on the list too. I recently fixed one depth sorting bug that Selden had alerted me about. But, if you're talking about depth sorting of transparent objects, that's a ways off.

I'm happy to wait for as long as it takes if it's still high on your list of priorities. (Actually I've no choice in the matter as I'm not in a position to fix these bugs myself )
JOOI: What's the technical hurdle that makes the transparent depth-sorting more difficult to fix than Selden's bug?
Secondly, to which category do the problems I reported to you earlier in the year belong to? (Do you remember those?)
Do you think those problems will be resolved by the bug-fix you mention above, or are they in the category of transparent object depth sorting bugs?

BTW. Do you have any rough gut feeling for the likely "performance hit" of solving the DS problems as compared to the performance hit of the rendering improvements in your recent threads? Is it likely to be more or less of a performance drag than these other improvements?

Regards
CC

PS. I apologise for harping on about DS issues when you've done all this great stuff on improving other aspects of the rendering, but just interested in progress on DS bugs as they cause a few serious problems as you know.
Personally, my approach (to software development in general) would be to give a higher priority to bugs such as the DS ones, which actually "break" the correct rendering of objects, versus quality improvements of things which aren't actually broken, which I would give a slightly lower priority to. (Of course I'm not doing any coding, but this is just my un-educated opinion from the point of view of the perceived overall quality of the software).
Of course, this opinion does not take into account the specific levels of difficulty and risk associated with these fixes, which I'm certainly not in a position to judge. Not to mention the fact that what you are doing is probably much more fun to code.

[t00fri]

Chris,

beyond what I wrote in the dev list, there is another gridlike artefact of the spec background visible now (it was there before, too, but MUCH weaker):

Have a look:



Bye Fridger

[t00fri]

Chris,

the clouds turning black (=> shadows) I was referring to in the dev list, arise if the "I" key is pushed. The white clouds are turned off, but their black shadows REMAIN . So it is a bug.

Bye Fridger

[jgrillo2002]

for me I have a GeForce FX 5500. will this work for my machine?

[selden]

jgrillo2002,

Yes.
[edit]
But you do have to remember to use the OGL 2 render path!
[/edit]

Chris,

Just to be difficult...

What's the chance that clouds could be persuaded to generate shadows onto adjacent clouds near the terminator?

[Telepath]

Oh boy Selden,

You're a perfectionist !!!

[PlutonianEmpire]

Unfortunately, it looks like I was mistaken about the 5nnn series of cards. The cloud shadows don't appear when using an Nvidia FX 5200 card. Apparently the v3 shaders are needed (GF 6nnn or better). 5nnn cards have v2 shaders

Sorry.

Aww, that sucks.

[selden]

I assume you're referring to my brief mistaken comment about 5nnn cards. I was wrong. I'd forgotten to switch to the OpenGL 2.0 render path.

Performance isn't all that great, though.
On a 2.4GHz P4 with an FX5200, I'm only getting about 6fps.

[PlutonianEmpire]

I assume you're referring to my brief mistaken comment about 5nnn cards. I was wrong. I'd forgotten to switch to the OpenGL 2.0 render path.

Performance isn't all that great, though.
On a 2.4GHz P4 with an FX5200, I'm only getting about 6fps.

Ah. I see. Well, it's a start.

EDIT: which texture size are you using? 2k cloudmaps?

[selden]

This speed is with the standard Celestria v1.4.1 final distribution. The only changes are that

a. I'm running a copy of the Celestia executable compiled this morning from the code in the SourceForge CVS repository and that

b. I raised the cloudheight from 7 to 70 so the shadows would be more visible.

[PlutonianEmpire]

This speed is with the standard Celestria v1.4.1 final distribution. The only changes are that

a. I'm running a copy of the Celestia executable compiled this morning from the code in the SourceForge CVS repository and that

b. I raised the cloudheight from 7 to 70 so the shadows would be more visible.

Oh, ok. Hopefully, it'll be faster in the final release.

[selden]

Here's what it looks like using Celestia's standard medres Earth and cloud textures:


Note the fps in lower left corner: 8.32

system:
512MB 2.4GHz P4, WinXP Pro SP2
128MB FX 5200, ForceWare v84.21
Celestia from CVS

[chris]

Chris,

beyond what I wrote in the dev list, there is another gridlike artefact of the spec background visible now (it was there before, too, but MUCH weaker):

This looks to me like it's caused by dithering in the cloud texture. Does it disappear completely when you turn clouds off?

--Chris

[chris]

Chris,

the clouds turning black (=> shadows) I was referring to in the dev list, arise if the "I" key is pushed. The white clouds are turned off, but their black shadows REMAIN . So it is a bug.

Oops! That was a silly oversight. I just checked in a fix for this problem. If you update the to latest CVS build, the cloud shadows will go away when cloud maps are disabled.

--Chris

[chris]

Here's what it looks like using Celestia's standard medres Earth and cloud textures:


Note the fps in lower left corner: 8.32

system:
512MB 2.4GHz P4, WinXP Pro SP2
128MB FX 5200, ForceWare v84.21
Celestia from CVS

I haven't had a chance to test the new code with a slower graphics card yet. I wonder if cloud shadows are the cause of the slowdown or if it's per pixel specular lighting. I do have some ideas for that may speed up the cloud shadow vertex shader.

Here's what the NVShaderPerf tool says about the performance of the bump mapping + specular pixel shader on a few different cards:

Code: Select all

Target: GeForceFX 5200 Ultra (NV34) :: Unified Compiler: v81.95
Cycles: 28 :: # R Registers: 4
Pixel throughput (assuming 1 cycle texture lookup) 28.57 MP/s

Target: GeForceFX 5900 Ultra (NV35) :: Unified Compiler: v81.95
Cycles: 20 :: # R Registers: 4
Pixel throughput (assuming 1 cycle texture lookup) 90.00 MP/s

Target: GeForce 6600 GT (NV43-GT) :: Unified Compiler: v81.95
Cycles: 11.00 :: R Regs Used: 3 :: R Regs Max Index (0 based): 2
Pixel throughput (assuming 1 cycle texture lookup) 363.64 MP/s

Target: GeForce 6800 GT (NV40-GT) :: Unified Compiler: v81.95
Cycles: 11.00 :: R Regs Used: 3 :: R Regs Max Index (0 based): 2
Pixel throughput (assuming 1 cycle texture lookup) 509.09 MP/s

Target: GeForce 7800 GT (G70) :: Unified Compiler: v81.95
Cycles: 9.00 :: R Regs Used: 3 :: R Regs Max Index (0 based): 2
Pixel throughput (assuming 1 cycle texture lookup) 1.07 GP/s


MP/s is megapixel per second, GP/s is gigapixels per second. Cycles is the number of clock cycles required to execute the shader, so the overall pixel throughput with a given shader (clock speed * number of parallel shader pipes) / cycles. You can see that in addition to having more shader pipes, the newer chips have lower cycle counts for the bump/specular shader. That's because they can process more than one operation per clock cycle (say a vector multiply/add, a dot product, and reciprocal square root.)

This version of NVShaderPerf doesn't support the 7900 GTX, but it's easy to extrapolate from the 7800 GT's 400MHz clock/20 shader pipes to the 7900 GTX's 650MHz clock/24 shader pipes to get a figure of 2.09 GP/s.

And if you're interested, here's the GLSL shader source generated by Celestia:

Code: Select all

 uniform sampler2D diffTex;
 uniform sampler2D normTex;
 uniform sampler2D specTex;
 varying vec2 diffTexCoord;
 varying vec2 normTexCoord;
 varying vec2 specTexCoord;
 uniform vec3 ambientColor;
 vec4 diff = vec4(ambientColor, 1.0);
 varying vec3 eyeDir;
 vec4 spec = vec4(0.0);
 uniform float shininess;
 varying vec3 lightDir0;
 uniform vec3 lightcolor0;
 uniform vec3 lightspecColor0;

void main(void)
{
    vec4 color;
    vec3 n = texture2D(normTex, normTexCoord.st).xyz * 2.0 - vec3(1.0, 1.0, 1.0);
    float l;
    vec3 eyeDirN = normalize(eyeDir);
    vec3 hv;
    float nDotHV;
    l = max(0.0, dot(lightDir0, n)) * clamp(lightDir0.z * 8.0, 0.0, 1.0);
    diff.rgb += l * lightcolor0;
    hv = normalize(eyeDir + lightDir0);
    nDotHV = max(0.0, dot(n, hv));
    spec.rgb += l * pow(nDotHV, shininess) * lightspecColor0;
    color = texture2D(diffTex, diffTexCoord.st);
    gl_FragColor = color * diff + texture2D(specTex, specTexCoord.st) * spec;
}


And here are the instructions that the shader compiler generates:

Code: Select all

MOVR  R0.xyz, f[TEX4];
ADDR  R0.xyz, f[TEX3], R0;
DP3R  R0.w, R0, R0;
RSQR  R0.w, R0.w;
MULR  R2.xyz, R0.w, R0;
TEX   R1.xyz, f[TEX1], TEX0, 2D;
TEX   R0, f[TEX2], TEX2, 2D;
MADR  R1.xyz, R1, {2, -1}.x, {2, -1}.y;
DP3R  R1.w, R1, R2;
MAXR  R1.w, R1, {0}.x;
DP3R  R1.x, f[TEX4], R1;
MAXR  R1.y, R1.x, {0}.x;
POWR  R1.w, R1.w, shininess.x;
MULR_SAT R1.x, f[TEX4].z, {8};
MULR  R2.x, R1.y, R1;
MULR  R1.x, R2, R1.w;
MULR  R1.xyz, R1.x, lightspecColor0;
MOVR  R1.w, {0}.x;
MULR  R0, R0, R1;
MULR  R1.xyz, R2.x, lightcolor0;
ADDR  R2.xyz, R1, ambientColor;
MOVR  R2.w, {1}.x;
TEX   R1, f[TEX0], TEX1, 2D;
MADR  o[COLR], R1, R2, R0;
END


--Chris

[selden]

Chris,

For what it's worth, enabling specularity on Mars resulted in about 11.6 fps with the same fov as when looking at the Earth (see my recent posting to the new features/specularity thread). With no specularity, I get about 20 fps. Since Mars has no cloud texture, presumably the cloud shader code isn't used.

[cartrite]

I compiled the latest cvs about an hour ago. These cloud shadows look great.



A couple of questions if I may.

This shot shows some lines that are along 180 west. Are they known? They disappear in a different rendering mode with shadows off.



They are also there with the default cloudmap.



This shot shows displaced shadows when using an 8k cloudmap. This may be caused by my graphics card or is there a 4k limit to the cloudmap size caused by my max tex size = 4k. My card only handles 4k textures and I'm wondering if the program is tiling the 8k map to bring it to the size my card can handle. Which I think was stated that the cloud shadows won't work with tiled maps even if they are tiled by the program at runtime. The 8k map works fine without shadows in a different rendering mode.



cartrite

[selden]

If you want to use a cloud texture larger than the size your card supports, then you have to set
CloudSpeed 0

As you wrote, Chris mentioned that shadows can't work for cloud VTs. I don't know if that restriction applies to oversized single image files.

[chris]

I should have put that on the list too. I recently fixed one depth sorting bug that Selden had alerted me about. But, if you're talking about depth sorting of transparent objects, that's a ways off.

I'm happy to wait for as long as it takes if it's still high on your list of priorities. (Actually I've no choice in the matter as I'm not in a position to fix these bugs myself )
JOOI: What's the technical hurdle that makes the transparent depth-sorting more difficult to fix than Selden's bug?

It's a fundamental limitation of all currently shipping graphics hardware. In order for translucent objects to be rendered correctly, the triangles must be sorted so that the ones furthest from the viewer are rendered first. You can do the sorting on the CPU, but for a moderately detailed model this will be very slow. There are no practical workarounds that completely solve the problem. Only partial fixes exist.

Secondly, to which category do the problems I reported to you earlier in the year belong to? (Do you remember those?)
Do you think those problems will be resolved by the bug-fix you mention above, or are they in the category of transparent object depth sorting bugs?

I'll have to look at your bug report again; there's a good chance that if it didn't involve translucent models, your problem may have been fixed by my change.

BTW. Do you have any rough gut feeling for the likely "performance hit" of solving the DS problems as compared to the performance hit of the rendering improvements in your recent threads? Is it likely to be more or less of a performance drag than these other improvements?

It's a more complicated discussion than I have time for right now But I do have a very good understanding of all the performance implications.

PS. I apologise for harping on about DS issues when you've done all this great stuff on improving other aspects of the rendering, but just interested in progress on DS bugs as they cause a few serious problems as you know.
Personally, my approach (to software development in general) would be to give a higher priority to bugs such as the DS ones, which actually "break" the correct rendering of objects, versus quality improvements of things which aren't actually broken, which I would give a slightly lower priority to. (Of course I'm not doing any coding, but this is just my un-educated opinion from the point of view of the perceived overall quality of the software).
Of course, this opinion does not take into account the specific levels of difficulty and risk associated with these fixes, which I'm certainly not in a position to judge. Not to mention the fact that what you are doing is probably much more fun to code.

It is more fun, but I've also spent time lately doing bug fixes, and will be doing some more once I've got cloud shadows working to my satisfaction.

--Chris