Post-1.6.0 performance

[t00fri]

Fridger,

I consider the 26% reduction in frame rate that I reported (117 fps --> 86 fps) to be a considerable slowdown, and that's with a graphics card which is much more capable than the average. (

But that was NOT a test using the standardized conditions of THIS thread!

How about investigating tests 1 and 2 of THIS thread with your low-end machine instead?
With my old FX5900Ultra I also don't see any evidence for a slow-down due to anisotropic filtering in Tests 1 & 2 (cf my numbers above).


Fridger

[chris]

Fridger,

I consider the 26% reduction in frame rate that I reported (117 fps --> 86 fps) to be a considerable slowdown, and that's with a graphics card which is much more capable than the average.

Exactly.

Granted, that test was with a Quadro workstation card and not a GeForce gaming card. I don't know what difference that makes in relative performance of anisotropic filtering. My tests here all have been using a gaming card with anisotropic filtering enabled. I'll make another test with it disabled to see how much it improves.

I don't know of any across-the-board difference in anisotropic filtering performance between the Quadro and GeForce series. There have been changes between various GPUs, with the cost of anisotropic filtering going up or down between GPUs. Various filtering optimizations have also been adopted and then sometimes later abandoned when the impact on rendering quality proved too noticeable.

I provided DOJOMO with a copy of Celestia compiled from the current svn code, but with both anisotropic filtering and SSE2 code disabled. He reported no improvement in performance.

You should provide an executable with anistropic filtering disabled and SSE2 *enabled*--his system has a Pentium 4M, which does support the SSE2 instruction set. It's possible (though I admit unlikely) that the lack of SSE2 is offsetting any performance gains from disabling anisotropic filtering.

For what it's worth, my personal interest is in whatever can be done to improve the performance when viewing many small objects simultaneously. Celestia's performance when viewing the control desk of the Hale Telescope Addon is one example. I'm currently getting only 10fps when viewing it

I need to dive into this with a profiler, but there should be some easy optimization opportunities here. I suspect that most of the time is spent doing expensive lighting calculations for all the different telescope components.

--Chris

[selden]

v1.6.0 / r4864+ai/ r4864-ai

191/202/202
111/157/158
210/204/211
123/116/117
162/165/168

So I see no appreciable difference in performance with or without anisotropic filtering when using a GF 7800 GTX. I'll do the Quadro test tomorrow (about 15 hours from now).

p.s.
I seem to be able to reproduce the 3% difference in the 3rd test (stars only).
I had "scaled discs" selected when running the first r4864+ai test,
but "fuzzy points" selected when running the r4864-ai test.
I get 204 when running the r4864-ai test with "scaled discs".

[t00fri]

There is an easy and clear way of precisely testing anisotropic filtering effects:

Take the Earth right after startup.

Then use the nvidia control utility to set successively AF = 0, 2x, 4x, 8x just for celestia.exe (i.e. on an application basis!)and watch the corresponding framerate.

Now this would be right for Celestia 1.6.0.

For SVN r4864 one can similarly vary the turned-on filtering rate in the code and quickly recompile each time. Then one can see also, whether there is any compensations from new code.

From the many such tests about anisotropic filtering I have done over time, my conclusion is: yes, with the old code, Celestia's fps drop by about 10-12% with my old, low-end card, if I vary JUST the amount of anisotropic filtering between 0 and 8x. In my view this is a relatively SMALL effect that can be uniquely tagged down with the described method.

If I do the same type of exercise with AA = 2x,4x,8x instead (with anisotropic filtering=0), the effect in my low-end machine is WAY bigger, like a slow-down by more than a factor of TWO around AA=8x!

Fridger

[selden]

Code: Select all

1.6.0/+ai+sse2/-ai+sse2/-ai-sse2/
1: 142 /  134 /    151 /    155 /
2:  75 /   81 /     85 /     87 /
3: 173 /  176 /    177 /    181 /
4:  80 /   76 /     79 /     78 /
5: 128 /  132 /    132 /    132 /


System:
2GB 1.86GHz Core2Duo E6300, Win XP Pro SP3
128MB Quadro 550, ForceWare 169.47
1600x1200 display, vsync disabled

With this test, I get only an 11% decrease in performance due to anisotropic filtering.
I guess filtering Saturn's rings takes more work .
It doesn't look like SSE2 makes any difference, though. The executable is slightly larger when SSE2 is enabled, so the compiler is certainly generating different code.

And, no, I didn't get the columns reversed. There seems to be a slight penalty when using SSE2, although I'm not sure if it's statistically significant.

[t00fri]

Code: Select all

1.6.0/+ai+sse2/-ai+sse2/-ai-sse2/
1: 142 /  134 /    151 /    155 /
2:  75 /   81 /     85 /     87 /
3: 173 /  176 /    177 /    181 /
4:  80 /   76 /     79 /     78 /
5: 128 /  132 /    132 /    132 /


System:
2GB 1.86GHz Core2Duo E6300, Win XP Pro SP3
128MB Quadro 550, ForceWare 169.47
1600x1200 display, vsync disabled

With this test, I get only an 11% decrease in performance due to anisotropic filtering.
I guess filtering Saturn's rings takes more work .
It doesn't look like SSE2 makes any difference, though. The executable is slightly larger when SSE2 is enabled, so the compiler is certainly generating different code.

And, no, I didn't get the columns reversed. There seems to be a slight penalty when using SSE2, although I'm not sure if it's statistically significant.

Selden,

sorry, if yesterday night I argued strongly that indeed we should do such tests right or --otherwise-- save our time and just follow our pre-existing qualitative views, as Chris did earlier...

Anyway, it is comforting that you precisely confirm my own findings (10-12% AIF slow-down only for low-end graphics).

Perhaps, I will find the time these days to benchmark the dependence on AIF in detail, ranging from 0x to 8x, say.

What is interesting furthermore, is that I also had found a slight slow-down with sse2 turned on, both for my low-end graphics (FX 5900Ultra) and for my new office machine (GX 9500GT/512 MB)

Fridger

[chris]

sorry, if yesterday night I argued strongly that indeed we should do such tests right or --otherwise-- save our time and just follow our pre-existing qualitative views, as Chris did earlier...

Um, pardon me? Fridger, can you please find a way to express yourself that is not insulting? Further testing (which I greatly appreciate) has born out the fact that anisotropic filtering causes slowdowns. It's a matter of opinion whether or not the roughly 10% performance drop is "significant" or not.

When doing further tests of anisotropic filtering performance, it's only necessary to test URLs #1 and #2. Anisotropic filtering is only used when applying texture to geometry; only a tiny fraction of the screen (stars and galaxy sprites) is covered with textured pixels in URLs #3-#5. Selden: do you still see a bigger performance drop from anisotropic filtering when viewing Saturn, or was that just a fluke?

--Chris

[t00fri]

sorry, if yesterday night I argued strongly that indeed we should do such tests right or --otherwise-- save our time and just follow our pre-existing qualitative views, as Chris did earlier...

Um, pardon me? Fridger, can you please find a way to express yourself that is not insulting?

--Chris

Chris,

sorry, my post is of course not meant to be insulting. Perhaps it read badly, since I tried to condense the point to 2 lines. As usual, I would be happy to switch to German, whence such unfortunate misunderstandings from my side would surely cease to exist

Here is a longer version of what I was trying to say:

Yesterday you drew conclusions about the importance of AIF in low-end machines,

On low-end systems, anisotropic filtering can slow down rendering considerably.

despite my insistence that the evidence you used was from another non-standardized benchmark in an earlier thread.

From that non-standardized benchmark, Selden claimed a substantial 26% of AIF-caused slow-down for his low-end graphics card. And you wrote

Exactly.

I did not comment, since you made me feel silly with your comment, despite my own existing tests, which I trusted of course.

Today, Selden seems to agree with my own findings, after doing the standardized benchmark instead. I therefore conclude (once again) that it is important to do such tests carefully and in an unbiased manner, before conclusions are to be drawn.

Fridger

[t00fri]

It's a matter of opinion whether or not the roughly 10% performance drop is "significant" or not.

--Chris

As you can read in my respective post of yesterday, i.e. before Selden's confirmation, I have always gotten a decrease of 10-12% due to AIF but NEVER more! There MUST be some slow-down without other compensating effects, of course. But 10% and 26% or more is a big difference!

If it is indeed your opinion that a mere 10% drop due to AIF is significant (, given all the benefits), then it's best if I drop out of this discussion...


Fridger

[selden]

I made a mistake when recording the previous benchmark values on this system. They were taken in the "OpenGL vertex program" render path, not the "OpenGL 2.0" render path.

Here are the values in the OpenGL 2.0 render path:

Code: Select all

1.6.0 / +ai+sse2 / -ai+sse2 / -ai-sse2/
1:  92.2 /  92.4 /  95.6 /  99.0
2:  32.3 /  32.0 /  32.4 /  32.6
3: 168   / 173   / 173   / 173
4:  81.5 /  76.7 /  79.3 /  78.6
5: 126   / 131   / 131   / 131


So anisotropic filtering is a negligible fraction of the total when frame rate is slowed by other computations.

Here's the URL that I used to test performance when viewing Saturn:

Saturn benchmark

This viewpoint shows Saturn with orbit paths disabled.
After obtaining that fps, I typed an "o" to turn on orbits.

These values are for the "OpenGL Vertex program" render path

Code: Select all

1.6.0 / +ai+sse2 / -ai+sse2 / -ai-sse2/ (orbits disabled)
  117 /     88.6 /      120 /     119
1.6.0 / +ai+sse2 / -ai+sse2 / -ai-sse2/ (orbits enabled)
 87.5 /     77.5 /     97.5 /    96.2


anisotropic filtering is 20 to 26% slower depending on whether orbits are enabled.
It looks like the SSE2 code might help a little when displaying Saturn and its shadows in this render path..

These values are for the "OpenGL 2.0" render path

Code: Select all

1.6.0  / +ai+sse2 / -ai+sse2 / -ai-sse2/ (orbits disabled)
  69.5 /     54.6 /     64.4 /     64.4
1.6.0  / +ai+sse2 / -ai+sse2 / -ai-sse2/ (orbits enabled)
  60.7 /     49.4 /    54.4  /    57.5


So in the OpenGL 2 render path, anisotropic filtering is 9 to 15 % slower depending on whether orbits are enabled.

In other words, when more computing is required for other effects, anisotropic filtering is a smaller fraction of the total. One can get quite fast frame rates when computing is minimized.

system:
2GB 1.68MHz C2D E63000, WinXP Pro SP2
128MB Quadro FX 550, ForceWare 169.47
1600x1200 display, virtical sync disabled

[chris]

If it is indeed your opinion that a mere 10% drop due to AIF is significant (, given all the benefits), then it's best if I drop out of this discussion...

I think that it is significant, and I'm concerned potentially larger drops on even older or lower end hardware. But, I never meant to imply that we should turn off anisotropic filtering: it gives a big improvement in rendering quality, and it should be enabled by default (as it is now) on hardware that supports it. The one possible point of contention is whether to implement a simple GUI control to allow user control over the maximum level of anisotropy. Given what we've seen so far, I would argue that we should have such a control as part of a 'Render Quality' panel.

--Chris

[t00fri]

If it is indeed your opinion that a mere 10% drop due to AIF is significant (, given all the benefits), then it's best if I drop out of this discussion...

I think that it is significant, and I'm concerned potentially larger drops on even older or lower end hardware. But, I never meant to imply that we should turn off anisotropic filtering: it gives a big improvement in rendering quality, and it should be enabled by default (as it is now) on hardware that supports it. The one possible point of contention is whether to implement a simple GUI control to allow user control over the maximum level of anisotropy. Given what we've seen so far, I would argue that we should have such a control as part of a 'Render Quality' panel.

--Chris

Chris,

I think that it is significant,

As you also noted, Celestia's fps display tends to run always through a range of values in a given configuration, that can already make up for a few percent of uncertainty...By convention we have agreed to always pick the peak values . Never mind.

On the other hand, I agree that one may contemplate a GUI panel for some user control of AIF and possibly also AA. If you feel you want such a panel (despite the lack of convincing data), it is certainly fine with me.

Personally, I would be hesitant, however, for two reasons:

• The NVIDIA control utility, for example, is already not the simplest piece of software for the uninitiated user. Notably, if one is out for adjusting BOTH global and process-based performance settings. The logical "cross" implications are not at all self-evident for the unexperienced.
  
  Now you want to interface this utility with another Celestia-based GUI panel that allows to set essentially the same thing, again on a process-based level (Celestia). For me this smells of problems...Why should I set AIF and AA in your advocated GUI panel, if I can set it for "celestia.exe" in the NVIDIA tool as well?
  Note that it exists for all popular OS and is automatically installed with the driver. I am sure there is a similar tool for ATI cards?
  
• such a panel once more adds to Celestia's complexity and one may well ask whether there is enough solid motivation from that point of view...Newbies will start asking what "anisotropic filtering" might be etc.

Fridger

[chris]

I made a mistake when recording the previous benchmark values on this system. They were taken in the "OpenGL vertex program" render path, not the "OpenGL 2.0" render path.

Here are the values in the OpenGL 2.0 render path:

Code: Select all

1.6.0 / +ai+sse2 / -ai+sse2 / -ai-sse2/
1:  92.2 /  92.4 /  95.6 /  99.0
2:  32.3 /  32.0 /  32.4 /  32.6
3: 168   / 173   / 173   / 173
4:  81.5 /  76.7 /  79.3 /  78.6
5: 126   / 131   / 131   / 131


So anisotropic filtering is a negligible fraction of the total when frame rate is slowed by other computations.

The atmospheric scattering shader used by the OpenGL 2.0 render path is complex, so the extra time required for the anisotropic texture sampling is hidden by all the extra calculation. The scattering shader is computation bound, whereas the texture unit is the bottleneck for the very simple shaders used in the other render paths. If you have a shader that compiles to instructions like this:

Code: Select all

tex2D
tex2D
tex2D


...it will be very sensitive when those tex2D instructions take longer because of anisotropic filtering. But, if the shader does lots of calculation, the shader compiler will schedule the arithmetic instructions to hide the extra latency of the anisotropic texture sampling. Something like this:

Code: Select all

tex2D
add
mul
tex2D
mul
mul
tex2D
sqrt
rcp


In a shader such as this, anisotropic filtering will be almost free...

These values are for the "OpenGL Vertex program" render path

Code: Select all

1.6.0 / +ai+sse2 / -ai+sse2 / -ai-sse2/ (orbits disabled)
  117 /     88.6 /      120 /     119
1.6.0 / +ai+sse2 / -ai+sse2 / -ai-sse2/ (orbits enabled)
 87.5 /     77.5 /     97.5 /    96.2


anisotropic filtering is 20 to 26% slower depending on whether orbits are enabled.
It looks like the SSE2 code might help a little when displaying Saturn and its shadows in this render path..

Interesting... I'm not sure what's happening here. It could be that anisotropic filtering is getting used a lot for ring texture when it's projected as onto Saturn as a shadow. At the moment, anisotropic filtering is turned on for every texture. Celestia could be smarter about that and only enable it when it's advantageous.

These values are for the "OpenGL 2.0" render path

Code: Select all

1.6.0  / +ai+sse2 / -ai+sse2 / -ai-sse2/ (orbits disabled)
  69.5 /     54.6 /     64.4 /     64.4
1.6.0  / +ai+sse2 / -ai+sse2 / -ai-sse2/ (orbits enabled)
  60.7 /     49.4 /    54.4  /    57.5


So in the OpenGL 2 render path, anisotropic filtering is 9 to 15 % slower depending on whether orbits are enabled.

On the GeForce 285 GTX, the OpenGL 2.0 path is actually fastest by a significant margin (1.6.0, 373 fps vs. 299 fps for the OpenGL vertex shader path.) The reason is reduced computation in the OGL 2 path. Saturn is actually rendered in two passes in the non-OGL2 paths: once for the base texture, and a second time for the ring shadow. I'd have expected a similar outcome (though less overwhelmingly in favor of OGL2) on your GeForce FX 550--not sure what's happening here, unless there's some additional OGL2 quality improvement that I've forgetting about.

--Chris

[Ricardo]

Why should I set AIF and AA in your advocated GUI panel, if I can set it for "celestia.exe" in the NVIDIA tool as well?
Note that it exists for all popular OS and is automatically installed with the driver. I am sure there is a similar tool for ATI cards?

As far as I know such tool is not available under osX.

[chris]

As you also noted, Celestia's fps display tends to run always through a range of values in a given configuration, that can already make up for a few percent of uncertainty...By convention we have agreed to always pick the peak values . Never mind.

On the other hand, I agree that one may contemplate a GUI panel for some user control of AIF and possibly also AA. If you feel you want such a panel (despite the lack of convincing data), it is certainly fine with me.

Lack of convincing data? But we do have data from you and Selden that unambiguously demonstrate a slowdown when anisotropic filtering is enabled. You may not be convinced that the slowdown is significant enough to justify having adding a graphics quality control panel, the performance drop is real.

Personally, I would be hesitant, however, for two reasons:

The NVIDIA control utility, for example, is already not the simplest piece of software for the uninitiated user.

... which is why I'd rather not force people to use it for simple things like enabling antialiasing.

Notably, if one is out for adjusting BOTH global and process-based performance settings. The logical "cross" implications are not at all self-evident for the unexperienced.

Now you want to interface this utility with another Celestia-based GUI panel that allows to set essentially the same thing, again on a process-based level (Celestia). For me this smells of problems...Why should I set AIF and AA in your advocated GUI panel, if I can set it for "celestia.exe" in the NVIDIA tool as well?
Note that it exists for all popular OS and is automatically installed with the driver. I am sure there is a similar tool for ATI cards?

• There's no NVIDIA control panel on the Mac. Reason enough by itself.
  
• The control panel only provides global control over anisotropic filtering. For optimal performance, we may want to enable anisotropic filtering for just some textures (e.g. just planet maps)
  
• It's easier for the user to never have to deal with the NVIDIA control panel and sort through a bunch of options that are of little or no relevance for Celestia. AA, anisotropic filtering, and a few Celestia-specific items could be configured on a single panel. This is better than directing the user to a separate app (that again may not even exist on their OS) to set some options (AA and texture filterng), while others are set inside Celestia.

[*] such a panel once more adds to Celestia's complexity and one may well ask whether there is enough solid motivation from that point of view...Newbies will start asking what "anisotropic filtering" might be etc. [/list]

A valid consideration... If we do add an anisotropic filtering option, there should be line of text explaining anisotropic filtering, e.g. "improves the appearance of planet textures, but may make Celestia slighly slower."

--Chris

[t00fri]

Lack of convincing data? But we do have data from you and Selden that unambiguously demonstrate a slowdown when anisotropic filtering is enabled. You may not be convinced that the slowdown is significant enough to justify having adding a graphics quality control panel, the performance drop is real.

Yes, sure, of course we agree on those O(10%) AIF-related slow-downs that become visible only on older/low-end machines. Clearly, I meant that for me O(10%) seems a rather marginal argument for a separate Celestia-based GUI panel.

• There's no NVIDIA control panel on the Mac. Reason enough by itself.
  

  Too bad. I got only two of our three supported OS (Win, Linux)
  
  

• The control panel only provides global control over anisotropic filtering. For optimal performance, we may want to enable anisotropic filtering for just some textures (e.g. just planet maps)
  

  
  That does sound interesting...
  
  

• It's easier for the user to never have to deal with the NVIDIA control panel and sort through a bunch of options that are of little or no relevance for Celestia. AA, anisotropic filtering, and a few Celestia-specific items could be configured on a single panel. This is better than directing the user to a separate app (that again may not even exist on their OS) to set some options (AA and texture filterng), while others are set inside Celestia.
  

  That strongly depends on the user and besides Celestia, there are other applications! I for example, employ the NV control heavily for a number of purposes related to Celestia AND otherwise. As a reminder, NV control allows to set lots of rendering options for each application separately, including Celestia. It does not allow, however, to set rendering options for parts of applications.
  
  But then coding all this looks like a major job, since AA for certain textures and color corrections might also be desirable ... Hmm, how about other higher priorities? I would know quite a few.

Fridger

[Reiko]

I notice no difference in look or performance when I have AiF turned on for celestia. It won't seem to work for me.

[Fenerit]

I notice no difference in look or performance when I have AiF turned on for celestia. It won't seem to work for me.

Do you not see things like these?

On default Mars texture:

No anisotropic filter:


Anisotropic filter 2x (minimum)


Note the pole's pinch reduction.

[t00fri]

I know about the benefits of AIF against the 'polar pinch' since a LONG time and use it correspondingly. Here is how the pole of Mars looks in my case. Not a bit of a pinch:



Fridger

[Fenerit]

Oh well, your "monster" texture is already without polar pinch when builded, I suppose; but the improvement on default one seem much more worthy, so far. At 8x it desapper at all, although I do not use that texture and 8x filter (this value is too hard to handle for my system).