My Globular Cluster Package for Testing

[t00fri]

Hi all,

as I announced 2 days ago in the Dev list, here comes my Globular Cluster package for general testing!

This package runs on my own development version since a fairly long time (together with quite a few further extensions ). For a first preview in May of this year, see my thread in the shatters.net forum:
viewtopic.php?f=10&t=12421&start=0

However, there are some important and more recent additions, to which I'll turn in a few moments.

I am planning to write a Blog or similar at our CelestialMatters site, entitled "Anatomy of a Globular Cluster Simulation", that exposes the interesting, non-trivial underlying theoretical framework in some detail. Throughout, I have managed to stay VERY close to Ivan King's successful scenario during the coding stage.

For general testing, I have generated a patch against the current SVN-4446 version. For VERY easy testing, I include as well a complete Windows executable archive via a setup.exe installer. Never mind, you can install and remove this package in parallel with your existing Celestia installation. The installation directory is called Celestia-glob and the version Celestia-1.6.0preF1.

Before turning to some technicalities, let me summarize some important new features:

• I managed to return consistently to a neat alpha-blending approach that I had originally started with.
  
  The blending is between a dominating single central "cloud" sprite at low resolution and many individual star sprites if the resolution becomes high enough. I had earlier discarded this approach, because of a complex 2d - 3d mixup problematics that I discussed in May,
  viewtopic.php?f=10&t=12421&start=0,
  when Chris L also suggested to use alpha blending.
  
  The trick that finally allowed me to succeed without compromising Ivan King's theory (!!), will be detailed in my forthcoming theoretical discussion at Celestial Matters.
  
  The blending now works very smoothly. Here is a little video for those who don't want to build the new code.
  
  (probably, you need to download the file and play it locally!!)
  Globular Cluster News, MPEG-4 AVI Video (20MB)
  
  
  
• While the globular color profile is still somewhat simplified, i.e generic, I modelled the colors pretty well from the best photo we got about globular clusters (M 80) from Hubble. Here is a direct comparison:
  
  First the best available true-color shot (Hubble):
  
  
  You nicely see the big orange "Red Giant" stars, as well as the "Blue Strugglers" further out...
  
  Next, the present Celestia rendering (I see that the image quality is deteriorated by strong rescaling!)
  
  
  Again, note the Red Giants and Blue Strugglers further out.
  
  Here is another larger-scale shot of 47 Tuc (top left) that is resolved already into stars and a smaller globular (bottom right) that is not yet resolved at this scale. 47 Tuc is the second brightest globular in the sky, bordering our SMC dwarf galactic neighbor that you see in the right half of the image.
  
  
  
  
  
• The globular data file, data/globulars.dsc, now includes ALL 150 officially accepted globular clusters in our MIlky Way galaxy. All scientific sources are listed in the boiler plate of my globulars.dsc file. Note that the included axis-angle data will eventually be unavoidable for implementing non-spherical cluster morphologies and orientation. Despite all clusters being still spherically symmetric, I used these angle-axis data already to map the globular frame to the skyplane. While this can be done also inside the code, it's a nice check for the forthcoming extensions...
  
  
• Last not least,( and actually quite tediously,) I implemented a SIMBAD-compatible cross-listing that I read out from SIMBAD via the SIMBATCH batch facility. The tedious part was an idividual truncation via Perl to max 4-5 alternative acronyms! So each globular display involves the most familiar 4-5 alternative acronyms that are strictly SIMBAD compatible. The respective result you see also displayed in the above illustrative shots and the video.
  
  That's a nice general and professional feature! Moreover, it's pretty, because clicking mouse-R -> info pops up a SIMBAD window, where you can check, compare and learn further things about each globular. That only works consistently, since all names now have SIMBAD compatible syntax!

As to the fading reddish globular labels, here is a shot from the asteroid IDA, together with plenty of globular clusters in view...


Now let me turn to some technical remarks:
-------------------------------------------

My package comprises quite a bit of new code, affecting many files.

Here is the list:
==================

• New files: globular.cpp, globular.h, toggle_globulars.celx
  toggle_globular_labels.celx, globulars.pl
  
• New data files: data/globulars.dsc
  
• Affected files: Makefile.am, cmdparser.cpp, deepskyobj.h,
  dsodb.cpp, render.cpp, render.h, simulation.h,
  celestiacore.cpp, celx.cpp, configfile.cpp, configfile.h,
  celestia.cfg.

The new code has been tested to build and work cleanly on Windows and Linux flavors, both with the traditional and the Qt4 GUI.

++++++++++++++++++++++
As to implementing the patch, I strongly suggest to start from a virgin SVN-4446 checkout of Celestia and patch my code with your favorite tool, starting from the Celestia-root level.
++++++++++++++++++++++

The new package involves a number intricate changes, like e.g:

In celestia.cfg, DeepskyCatalog -> DeepSkyCatalogs is now an array and accommodates more than one DSO data file name! My galaxy data file data/deepsky.dsc should eventually be renamed into data/galaxies.dsc, since the new globular data are correspondingly located in a file data/globulars.dsc. So it now looks like so in celestia.cfg:

Code: Select all

 
   DeepSkyCatalogs              [ "data/deepsky.dsc"
                                  "data/globulars.dsc" ]     


This example may serve as a warning illustration that you might easily encounter problems if you start "fiddling" rather than implementing the full patch at once...

Since I didn't want to touch the various GUI's or the key shortcuts at this stage, I included two trivial celx scripts, called toggle_globulars.celx and toggle_globular_labels.celx. They just serve to toggle ON|OFF the display of the globulars and their labels (which are of course of "auto-fading" type).
You can also simply include these two new celx functions into your start.celx, in case you are using it:

celestia:show("globulars")
celestia:showlabel("globulars")

celestia:setfaintestvisible(7.0) <== recommended in addition

So you should build Celestia with LUA activated and execute these scripts if you don't see some nice clusters with redish globular labels (M 4, Omega Centauri, 47 Tuc, M 53, M 2, ...) in the OGL canvas.

The Perl script (globulars.pl) that I wrote to extract the globular data (globulars.dsc) of ALL known globulars, you find in src/tools/globulars.

Here are some brief operating instructions:
++++++++++++++++++++++++++++++++++++++++++++++

Enter (M 4, Omega Centauri, 47 Tuc, M 53, M 2, ...) in the command line, hit RETURN and then center the globular with the 'C' key. If the particular label|globular is not visible already then the globular is dimmer than magnitude ~ 6.

Make sure that the automag feature is activated (CTRL+Y)!!!

The best way of enjoying globulars is similar to galaxies: Move your mouse forward (backward) while pressing SHIFT + mouse_left (zoom effect). At small resolutions you just will see a fuzzy blob, which eventually resolves into a beautiful cluster with many individual stars, when the zoom level (SHIFT + mouse_left)) is high enough.

You can also hit the key G=goto and then use a mixture of CTRL+mouse_left movements (distance change) and SHIFT+mouse_left (zooming). The above video uses precisely these two key sequences!
+++++++++++++++++++++++++++++++++++++

Finally, here is the patch for download:
+++++++++++++++++++++++++++++++++++++++++++++++

Zipped Dir with 4446 patch file and patch.log

and the complete Windows Package with Installer for "quick click testing"
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Windows Executable with Installer

NOTE: After patching a fresh SVN-4446 checkout, the Windows project files have still to be modified to account for the two new files: globulars.cpp, globulars.h. Since I am using the commercial VC 2003.net compiler, I could not do this for you!

For Linux (KDE) just run in the celestia root dir:

> patch -p1 < diff-4446.txt

I have included a patch-4446.log file, whence you see that there should be NO warnings or worse...

> make -f Makefile.cvs
> ./configure --prefix=/usr/local --with-kde --with-lua (etc. spice...)
> make
> make install (don't overwrite your existing installation!!!)
++++++++++++++++++++++++++++++++++++++++++++++++++

Let me know what you think.

Enjoy,
Fridger

[chris]

I think that the globulars look damned nice under a lot of viewing situations. The blending of the 'cloud sprite' to the many stars representation looks good, though it may be a bit too abrupt. I suspect that this is due to the fact that sprites used for the stars tend to disappear when they get to be less than one pixel wide.

At high zoom factors, the globulars start to look unrealistic:

m13-zoom.jpg

The blockiness appears to be a result of the sprite texture brightness not going to black at the edges. That should be an easy thing to fix.

The star colors look odd, particularly the green. I imagine that in aggregate, the sprites reproduce the spectral profile of the globular cluster accurately. Indeed, at fields of view greater than 1 degree, I didn't notice the green stars at all. At greater zoom factors, the colors become distracting. Would it be possible to restrict the star colors to blackbody colors and still have an accurate overall appearance for the cluster?

Lastly, the sprites are much too big. It would be nice if we could get them to behave more like stars, which remain point like even at high zoom factors. Unfortunately, the actual star rendering code is probably too specialized for rendering the star database. Without significant modifications, it would be difficult to use it for globular rendering. But maybe you could simply limit the sprite size to a few pixels? In order to limit the sprite size on screen, the sprite geometry could be clamped to some value determined by the field of view and distance to the viewer.

--Chris

[chris]

The new package involves a number intricate changes, like e.g:

In celestia.cfg, DeepskyCatalog -> DeepSkyCatalogs is now an array and accommodates more than one DSO data file name! My galaxy data file data/deepsky.dsc should eventually be renamed into data/galaxies.dsc, since the new globular data are correspondingly located in a file data/globulars.dsc. So it now looks like so in celestia.cfg:

Code: Select all

 
   DeepSkyCatalogs              [ "data/deepsky.dsc"
                                  "data/globulars.dsc" ]     


A useful change . . . Certainly we do not want to be forced to include globulars and galaxies all in the same data file.

--Chris

[t00fri]

I think that the globulars look damned nice under a lot of viewing situations. The blending of the 'cloud sprite' to the many stars representation looks good, though it may be a bit too abrupt. I suspect that this is due to the fact that sprites used for the stars tend to disappear when they get to be less than one pixel wide.

At high zoom factors, the globulars start to look unrealistic:

m13-zoom.jpg

The blockiness appears to be a result of the sprite texture brightness not going to black at the edges. That should be an easy thing to fix.

The star colors look odd, particularly the green. I imagine that in aggregate, the sprites reproduce the spectral profile of the globular cluster accurately. Indeed, at fields of view greater than 1 degree, I didn't notice the green stars at all. At greater zoom factors, the colors become distracting. Would it be possible to restrict the star colors to blackbody colors and still have an accurate overall appearance for the cluster?

Lastly, the sprites are much too big. It would be nice if we could get them to behave more like stars, which remain point like even at high zoom factors. Unfortunately, the actual star rendering code is probably too specialized for rendering the star database. Without significant modifications, it would be difficult to use it for globular rendering. But maybe you could simply limit the sprite size to a few pixels? In order to limit the sprite size on screen, the sprite geometry could be clamped to some value determined by the field of view and distance to the viewer.

--Chris

Chris,

for each of the points you raise, there are intuitively defined, hard-coded parameters that I have introduced for tuning purposes. Their meanings are commented precisely in my code. Altogether there are detailed comments in the code.

If you want a slower/later blending, increase P1 from 65 pixels to 95, say.

I suspect that this is due to the fact that sprites used for the stars tend to disappear when they get to be less than one pixel wide.

No I don't think so, since I have explicitly decreased 'minimumFeatureSize' by a factor of TWO in renderGlobularPointSprites().

The color profile may be easily tuned via well commented parameters.

The sprite sizes and their range are also set explicitly.

Color adjustments are not easy to do /objectively/, since analog and TFT monitors have a quite different color rendering profile. I worked with both in parallel and found it hard to arrive at a universal setting. Same with sprite sizes.

Moreover, the globulars (like my galaxies) are NEVER meant to be looked at at CLOSE distances. So I don't see what these extreme zooms are to mean physics-wise that you present?

As I wrote in various places, the present color profiles are only /generic/ and will be replaced by a quantitative, data-supported ones eventually.

Fridger

[t00fri]

@color again:

eventually this sort of (color-magnitude) info will be implemented directly from the respective catalog entries.


Then different clusters will have different color profiles that match the measurements!

I was already investigating about using a black-body spectrum. This is still in progress. Eventually, I want to visualize the precise catalog info in first priority. Photos always are too much enhanced relative to visual or small telescope observation, to make them look spectacular...

Fridger

[chris]

I think that the globulars look damned nice under a lot of viewing situations. The blending of the 'cloud sprite' to the many stars representation looks good, though it may be a bit too abrupt. I suspect that this is due to the fact that sprites used for the stars tend to disappear when they get to be less than one pixel wide.

At high zoom factors, the globulars start to look unrealistic:

m13-zoom.jpg

The blockiness appears to be a result of the sprite texture brightness not going to black at the edges. That should be an easy thing to fix.

The star colors look odd, particularly the green. I imagine that in aggregate, the sprites reproduce the spectral profile of the globular cluster accurately. Indeed, at fields of view greater than 1 degree, I didn't notice the green stars at all. At greater zoom factors, the colors become distracting. Would it be possible to restrict the star colors to blackbody colors and still have an accurate overall appearance for the cluster?

Lastly, the sprites are much too big. It would be nice if we could get them to behave more like stars, which remain point like even at high zoom factors. Unfortunately, the actual star rendering code is probably too specialized for rendering the star database. Without significant modifications, it would be difficult to use it for globular rendering. But maybe you could simply limit the sprite size to a few pixels? In order to limit the sprite size on screen, the sprite geometry could be clamped to some value determined by the field of view and distance to the viewer.

--Chris

Chris,

for each of the points you raise, there are intuitively defined, hard-coded parameters that I have introduced for tuning purposes. Their meanings are commented precisely in my code. Altogether there are detailed comments in the code.

If you want a slower/later blending, increase P1 from 65 pixels to 95, say.

I suspect that this is due to the fact that sprites used for the stars tend to disappear when they get to be less than one pixel wide.

No I don't think so, since I have explicitly decreased 'minimumFeatureSize' by a factor of TWO in renderGlobularPointSprites().

I wasn't referring to any issue with your code, but rather to the fact that triangles less than one pixel in size are likely to simply disappear when rasterized. I confirmed that this was the case by enabling 8x antialiasing and observing that the transition becomes smoother. This is because subpixel rasterization allows the sprites to fade out rather than disappear abruptly. With 8x antialiasing enabled, a tiny sprite may cover from zero to eight subpixels, and its brightness will be scaled accordingly with pixel coverage. Without antialiasing, the sprite covers one pixel (if the sprite overlaps a pixel center) or no pixels (if it falls between pixel centers), resulting in abrupt appearance/disappearance of sprites. It could be solved by enforcing a minimum sprite size and fading small sprite brightness in software. But, I don't think it's such a serious issue. You should still however do the experiment of comparing the antialiased and aliased zooming side by side.

The color profile may be easily tuned via well commented parameters.

To my eye, the overall color profile already looks quite good. My only issue is with the colors of the individual stars.

The sprite sizes and their range are also set explicitly.

Color adjustments are not easy to do /objectively/, since analog and TFT monitors have a quite different color rendering profile. I worked with both in parallel and found it hard to arrive at a universal setting. Same with sprite sizes.

Moreover, the globulars (like my galaxies) are NEVER meant to be looked at at CLOSE distances. So I don't see what these extreme zooms are to mean physics-wise that you present?

I don't think that this limitation needs to exist for globulars. My point is that a fairly simple change--clamping sprites to some maximum screen size--could dramatically improve the appearance of globulars at close distances. I should have a chance to experiment with the code tomorrow; I'll try and demonstrate what I mean (unless you beat me to it . . .)

--Chris

[cartrite]

Nice Job Fridger!!!!!!!!!!!!!!!!
I haven't seen any problems yet. The globular clusters look fantastic. At a distance, they look almost real.

m-13-1.jpg

m-13-2.jpg

m-13-3.jpg

cartrite

[t00fri]

Thanks Steve,

good to know also that the Windows executable seemed to have worked. My only concern here is that people without installed compilers might be lacking one of the required Microsoft *.dll's. All the others I have included of course.

Fridger

[cartrite]

Thanks Steve,

good to know also that the Windows executable seemed to have worked. My only concern here is that people without installed compilers might be lacking one of the required Microsoft *.dll's. All the others I have included of course.

Fridger

OOPS, I didn't try the windows executable yet, I built the svn 4446 on Linux KDE. I'll give windows a try later. Got some ISIS programs running on Linux.
cartrite

[chris]

Thanks Steve,

good to know also that the Windows executable seemed to have worked. My only concern here is that people without installed compilers might be lacking one of the required Microsoft *.dll's. All the others I have included of course.

Fridger

OOPS, I didn't try the windows executable yet, I built the svn 4446 on Linux KDE. I'll give windows a try later. Got some ISIS programs running on Linux.
cartrite

I did test the Windows package and it worked fine. However, I've got two different Microsoft compilers installed.

--Chris

[t00fri]

Chris,

• the blockiness at high zoom factors that you noticed was of course a trivial oversight ==> fixed.
  

• I don't think that this limitation needs to exist for globulars. My point is that a fairly simple change--clamping sprites to some maximum screen size--could dramatically improve the appearance of globulars at close distances. I should have a chance to experiment with the code tomorrow; I'll try and demonstrate what I mean (unless you beat me to it . . .)
  
  --Chris
  

  Well, at least until HDR is the acting standard, personally, I would never want to allow a close approach to stars of any kind, including the ones from the globulars. I am suffering since years when I see Celestia's suns at close distance
  
  Therefore --originally-- I did not even spend a minute on trying to improve the close-by appearance.
  
  But anyway, I did a quick hacking in Globular::renderGlobularPointSprites() now, which indeed gives a quite pleasing journey through the globulars :
  Best to set some high speed e.g. towards M 4 (CTRL+F7, hit A a few times) and watch what happens....
  
  

  Code: Select all

  Point3f relPos = p + offset;
float screenFrac = size / relPos.distanceFromOrigin();         
         
float size_hold = size;
if (screenFrac >= 0.006f)   
    size = 0.006f * relPos.distanceFromOrigin();
            
glTexCoord2f(0, 0);          glVertex(p + (v0 * size));
glTexCoord2f(1, 0);          glVertex(p + (v1 * size));
glTexCoord2f(1, 1);          glVertex(p + (v2 * size));
glTexCoord2f(0, 1);          glVertex(p + (v3 * size));       
size = size_hold;   

  
  I had to save the unclamped value of 'size' into 'size_hold', since for various other reasons I finally did not sort the sprites wrto distance from the center.
  
  I'll do a little video now.
  
  
• Above, you obviously examined the individual colors by zooming (SHIFT+Left drag) NOT by really approaching (CTRL+Left drag). That's why one could see also some distant blue "blue stragglers" near the zoomed center, which looks disturbing.
  
  However, if you really travel to the cluster you will see only colors that make sense and look quite OK.

Fridger

[ANDREA]

I used Fridger's Win installer, and here the results, poorly shown here due to the heavy size reduction (the original was 1920x1200) and 7/12 quality to reduce the image dimensions.
All of them show the Omega Cen globular that looks... well, outstanding!
Thanks a lot, Fridger, the wait was worth the results!
Bye

Andrea

[t00fri]

Chris ...and whoever wants to travel right through a Globular ,

Here is a little (and quite nice ) video that travels you right through the center
of M 4 after I incorporated my above patch:

Speeding through M 4's Center [MPEG-4, AVI, 18.3 MB, 720x480 video)!
(VLC plays it on any OS, Windows Media Player is OK as well)

You may again have to first download the .avi file for smooth performance. The video also illustrates that the blending is quite smooth. The video's color rendering is rather meager, though.

Enjoy,
Fridger

[t00fri]

Many thanks for testing, ANDREA. Good that things seem to work fine...

Fridger

[ANDREA]

Many thanks for testing, ANDREA. Good that things seem to work fine...Fridger

No, thank you for giving us this new piece of scientific rigour in Celestia, moreover VERY nice to look at.
I'm sure that the future releases, if any, will be even more enjoyable, but IMO this one is already very satisfactory.
BTW, I just looked at your last movie, very nice.
Here I'm showing the globular clusters cloud around the Milky Way.
Well done, appreciated.
Bye

Andrea

[t00fri]

Excellent, ANDREA.

So did you have to download the video first? Shatters is a bit slow right now (downloads?) .

Fridger

[Vincent]

Fridger,

Your globulars look indeed amazing!

I found an issue when looking at them at close distance, though.
This might be related to the issue Chris has pointed out (sprites looking like squares):

[t00fri]

Fridger,

Your globulars look indeed amazing!

I found an issue when looking at them at close distance, though.
This might be related to the issue Chris has pointed out (sprites looking like squares):

Thanks Vincent,

but I really don't see sprites looking like squares on your or my screenshots. The central agglomeration of sprites in your screenshot may accidentally look a bit square, but that's not generic.

But because a trivial oversight, there was a certain blockiness at largest zoom factors that I fixed already.

Fridger

[Vincent]

but I really don't see sprites looking like squares on your or my screenshots. The central agglomeration of sprites in your screenshot may accidentally look a bit square, but that's not generic.

Fridger,

The square-sprites I see are located at the periphery on my screenshot:

[t00fri]

Chris,

as you see, I am responding step-by-step to your comments:

Like this one:

The star colors look odd, particularly the green.

Well I don't think they are looking "odd" since the original 1777x1864 true-color TIF of M 80 has them, too:

Here are a few green stars from the Hubble photo that have precisely the same RGB composition as my green stars:



I have quite a bit of practice in transfering color profiles from photos /quantitatively/ to my code....Just compare the green color in Vincents screenshot above.


Fridger