Some architectural issues

[doctorjoe]

Here are some architectural issues with celestia that I'd like to bring up

* Bodies and stars are handled differently - There is a lot of rendundant code because stars and bodies are handled differently. All of the bodies in celestia should be subclasses of a single Body class

* Much too much dispatching. Body has an internal tag which is used all over to dispatch to different behavior. This should be replaced by subclassing

* Planetary systems assume that there is a primary body. The way things should work is that there should be a "BoundSystem" class which can allow objects of roughly equal mass around a central barycenter. This abstraction is probably essential for binary or multiple star systems.

* Math system requires too much casting. You should be able to do

Point3d x;
Point3f y = x;

and its easy to add templates to make this happen.

Also Vector and Points are really the same thing, and it would simplify the code considerably if they are combined.

* UniversalCoord should really be implemented as Point3<Bigfix>. If you have the template right you can do things like

UniversalCoord a, b;

Point3f = a - b;

Also, you should also be able to have quaternions containing BigFixes, and with the right templates these should interoperate with other elements.

* Create a Distance class so that units of measure are handled automatically

* Celestia really needs to handle precession of the axes.

* The render show/hide configuration functionality really should be abstracted and put into its own class.

[doctorjoe]

I meant that PrimarySystems should *not* automatically assume that there is a primary body.

[Sirius]

Bodies and Stars: The original issue with stars is that there are literally millions of them. Even if subclassing takes only four bytes per star as additional type information (original argument made by chris somewhere in the code or doc) the overhead would be significant to say the least. Additionally, the stars are organized in an octree that is built on startup, unlike other objects. If you want to generalize (which would in most circumstances be a Good Thing, in this case it needs thought) I propose thinking especially about the memory structure of all the objects. If you work with High-Level languages all the time, you tend to forget (i do, anyway :-) ) that all these things are still bytes in memory. And in a high-memory-consuming application like celestia, this makes the difference between swapping and not swapping...
Enough of the sermon :-)

I'd rather like to have the memory structure of celestia in a wiki to play around with...

Apropos dispatching: How well can you work with RTTI in C++?

With all these Bigfix types, also you have to watch your memory consumption. As I see it, only the user's position is in Bigfix right now. Handling measures automatically is propably a good idea, but still, unit signatures would again eat up a lot of precious memory (there are a lot of distances in celestia...)

Handling of precession: Yes

Another point: Using XML instead of SCC? There seems to be a started effort in the source... *ducks from those who have written SCC editors*

Greetings, Sirius

[Spaceman Spiff]

On the Bodies and Stars issue, I think doctorjoe's idea is not incompatible with conserving memory or keeping star searching up to speed with an octree. I'd agree with him.

For those interested, the star database overhead problem is mentioned in the DEVGUIDE.txt doc that Fridger kindly pointed me to in the celestia root directory of a CVS snapshot.

I'll use something I wrote in topic Physics and Astronomy ?» Light scattering in earthlike atmospheres? (http://www.celestiaproject.net/forum/viewtopic.php?t=4714) to illustrate why what doctorjoe wrote can work:

We currently have too little control over defining stars in Celestia to make sensible multiple star systems. Initially, we had to pick up whatever Hipparcos gave us in stars.dat, but this usually gave single star systems. Then we had the Tycho catalogue extend stars.dat, to include wide systems but it has two problems: a) wide binaries/multiples consist of 'fixed' stars the cannot move in orbits; b) entire systems that are close binaries are usually missing - why? - because the stars' fast orbital motions caused Hipparcos to read their positions with larger errors and the catalogue bad-parallax filter rejected them for it. Alula Australis was a perfect example - a naked eye star that 'disappeared' with the Tycho-based stars.dat for being a naughty pair of close binaries. Then we got STC files, but addition of companion stars using STC files still leaves them fixed in space, and addition of stars as 'emissive' planets in SSC files is, well, ugh. Even then, adjacent planets still have a single light source appearance.

So, my proposal on the first step:

We reduce each star system to a single 'invisible' class reference marker to be some median position for that system (e.g., the 4 or 5? stars of the Alula Australis system). To do this, we need to process all stars in the catalogue to derive a median point for each star system - that's the hard bit: Alpha Cen A and B obviously belong together, but do we for example tie Alcor to Mizar or not? Maybe distant companions are better left to drift under proper motion instead.

In the absence of any STC files to define that system's stars, Celestia just uses the Hipparcos/Tycho catalogue 'default' star(s) to render in the sky. Otherwise, if there's a STC file bearing the star system name or HIP name, it takes over. Note that there's no need to consider this reference marker until we 'enter' the system (at whatever 1000 A.U.'s that is), so for distant systems in the sky, we just render the star according to the good old-fashioned catalogue.

The reference marker won't end up being the true barycentre, so we need to be able to specify its offset from that - this can be worked out by knowledge of masses and positions of stars as observed for visual binaries, or totally made up for spectroscopic, hypothetical, fantasy or simply unknowable systems. It may be some tricky 3D geometry, but I'm sure enthusiats will start building up these STCs like people make Add-Ons for anything else. From this barycentre, we can reference the usual Celestia orbits - the kinematic (Keplerian) kinds - and reference further barycentres of further binaries as being in orbit about prior barycentres (like what you did with the Nearby Stars STC/SSC files, Grant). All barycentre, star, and star orbit stuff goes into this STC file.

Then, when we've got the stars in the STC file in the right orbits, we add the planets in SSC files. Planets either orbit a star or a pair of stars' barycentre at a distance, maybe even the main barycentre, maybe even double planets have their barycentre orbit another barycentre. The possibilities are endless! Reference plane for orbit inclination may need a bit of thought.

That, I hope, would at least lay the foundation for multiple star systems in Celestia, even before we start on working out the resultant multiple-source lighting problem. There's also the Roche-lobe star shape problem for close/contact binaries - Beta Lyrae, here we come!

... and to my delight, Chris responded:

The implementation of multiple star systems that I've been playing around with is a lot like what Spiff described.

... and we got to manipulate stars in STC files as well. Selden posted details on Page 2 of topic Celestia Users ?» Celestia 1.4.0 prerelease (http://www.celestiaproject.net/forum/viewtopic.php?t=6067).

.. and Lo! It was to be in Celestia 1.4.0pre* ever since.

If what doctorjoe is proposing were followed, we could abandon the STCs and the *.stc file extension altogether. This would be good, because the .stc extension unfortunately clashes with the OpenOffice spreadsheet template file extension.

So, we should replace what are referred to as 'stars' in the octree with stellar system 'placeholders'. That's where a new C++ class comes in. All placeholders are rendered as the usual pointlike stars according to the Hipparcos/Tycho catalogue data when outside the current stellar system, because that's how they appear from far away. Upon entering a system, Celestia checks whether there is an existing SSC, if not, it uses that catalogue data to guess the appearance of the star as it does now, otherwise as the star(s) is (are) defined in the SSC. You can form a heirarchy: a main barycentre hangs off the placeholder, the main star(s) hang off that, further stars or planets hang off those, further planets or moons hang off those.

N.B. ideally, the stellar system barycentre is at the placeholder position, or rather, the placeholder is positioned where the stellar system placeholder should end up. That relies on knowing precise masses for real multiple star systems to work out where that barycentre is compared to the stars' current catalogue positions. It might mean someone 'processing' the entire database from 'stars' to 'system placeholders': there's not necessarily a one-to-one correspondence. That's why I talk of a need for a barycentre offset in my quote above.

Another advantage of including stars as bodies inside SSCs is that their orbits will now be drawn, for example, the orbits of Alpha Cen A and B are currently not, nor are those for barycentres. Barycentres should also be a subclass of Body, yet invisible, but with orbits.

Some consideration might need to be made for wide multiple star systems visible from current stellar system. I do not know whether any nearby stars (say within 10 light years of current observer position) can be checked from their SSC files for all bodies that are stars, and rendered from that, and whether this could be done while travelling, or should wait until a stellar system is properly entered.

It seems to me that doctorjoe is very compentent at C++, and my understanding from here (http://www.cplusplus.com/doc/tutorial/) is that it is a low-level language and the things doctorjoe says seem to be exactly those recommendations made about how to write C++ for best system/memory resources usage. I think following such ideas would be a good way forward.

Of course, abandoning STCs would be more likely to create reverse imcompatibility than class architecture changes (I think?), which means a Celestia version 2.0 step. This does not mean a fork, but a fork could bring this about.

On the other hand, silly me, that would remove any way of creating fictional star systems.

Perhaps those SSC editors could be revised to read in SSCs, add stars and export SSC and/or XML? Also output a little delta placeholder database for fictional add-ons? No need to duck then, Sirius?

Just my two eurocents.

Spiff.

[Toti]

* Math system requires too much casting. You should be able to do

Point3d x;
Point3f y = x;

and its easy to add templates to make this happen.

Also Vector and Points are really the same thing, and it would simplify the code considerably if they are combined.

I agree. Much code is just there to make these promotions take place. It could be handled by adding a few operators in the corresponding math templates. Code would be much more readable, but it might also introduce some subtle bugs as conversions are done "under the hood".
The Point<> and Vector<> duality is common in 3d programs. It improves the abstraction level but it can be annoying also. Some conversions are usually automatic eg. Point - Point = Vector, Point + Vector = Point, but others are much less intuitive.

[Paolo]

This post seems a bit a developer wishlist.
So I'll add my opinions.

As we are talking about SSC file formats, I would like to rewrite the entire datamodel (the XML file format should be a consequence of this). The main change that I would like to introduce support for add-on manager.

There are some other posts in this forum about the argument but now I'm in a hurry so I'll not attach the link to them.

[t00fri]

In my view, the arguments put forward above about beautifying|simplifying the code, don't strike me as particularly urgent at this (difficult) time for Celestia development. Also rewriting major parts of the core code for whatever reasons has subleading priority in my view...

I think, it is crucial NOW to get some new projects under way or to fix old bugs.

Doctorjoe did a few nice fixes already. I particularly liked his removal of the very old orbit bug! Perhaps, there is also a chance of closing the /very old/ holes in planetary skies at last . Also his very general approach to arbitrary coordinate system grids and readouts I find just GREAT. Both I and Chris have been thinking precisely along such a general approach for years without getting around to actually implementing it!

Toti, I and also Chris are working (quite intensively, in fact) to get the 10000+ galaxy rendering project off the ground. At least some people are working on it and exchange plenty of mail (Hi Toti, hi Chris ).

If there are some "hidden" yet interested OCTREE culling experts in the forum, please get into contact!!

So...optimism is IN

Bye Fridger

[doctorjoe]

I've already created some template changes to handle automatic format conversions. It's the file template.patch in my extensions patches.

3d graphics tend to make a distinction between points and vectors. Physicists don't, and the argument for not making this distinction in celestia are the places where you force a type conversion. The one place where celestia actually does treat points differently from vectors is the matrix transformation.

Also, I'd take the abstraction with binaries a bit further. Satellites and planetary systems should be represented by the "placeholder." After all, the moon does not revolve around the center of the earth, but rather around a common barycenter which is quite distant from the center. (The big issue with binary stars is an interesting one in that we don't know the orbit in three dimensions for the vast majority of the them. Displaying ellipticals is also going to be interesting since we don't know what any given elliptical likes like in 3-d.)

I disagree about the necessity for this sort of utility coding. Without these sorts of cleanups, it becomes much more difficult to make future improvements cleanly. For example when dealing with galaxies, you really want to deal with either Kpc, Mpc or Z. This is going to add a lot of complexity if you don't type the distances, but if you had a distance class, the addition would be trivial.

[t00fri]

doctorjoe,

of course, I agree with you in principle! Being a (theoretical) physicist myself, I suffer from the untransparent way units are handled in Celestia since 3 1/2 years of being part of the development team...

Knowing also a bit what's going on "in the background", I just feel that concentrating on some exciting and /visible/ issue is simply better right now. If you are willing to work on "utility" coding issues, that's certainly useful too, notably in the long run.

Bye Fridger

PS: Sorry, since this afternoon the forum has again become intolerably slow for me. My abilities to communicate are getting close to zero!

I wrote again an email to Chris, but from experience, I suppose he is again out climbing over the weekend...

[Spaceman Spiff]

Also, I'd take the abstraction with binaries a bit further. Satellites and planetary systems should be represented by the "placeholder." After all, the moon does not revolve around the center of the earth, but rather around a common barycenter which is quite distant from the center. (The big issue with binary stars is an interesting one in that we don't know the orbit in three dimensions for the vast majority of the them. Displaying ellipticals is also going to be interesting since we don't know what any given elliptical likes like in 3-d.)

Ah careful, doctorjoe! What I meant is that a stellar system (such as the 4 stars of Alula Australis) should be represented by one placeholder in the stars database. The first barycentre between the two pairs of star in that system may co-incide with the placeholder, or may be offset. The next two barycentres would orbit that first barycentre, and each pair of stars would orbit their respective barycentres, the whole system being a good ol' Keplerian approximation of orbits. Placeholders and barycentres would different objects, the point being that placeholders correspond to what's in the stars database, derived from the Hipparcos/Tycho databases, and barycentres correspond to SSC files. The reason is that Celestia can skip a lot of unnecessary work if it can draw Alula Australis as a point-like object in the sky based on the concise info in the star database when the observer is far (> 1 light-year?) from that system. I choose Alula Australis also to caution that visual binaries such as this often have separate entries for each component, and the database would need to ''merge' them back into the single point-like appearance the system has from such distance.

For the Solar System, VSOP87 takes care of baycentric motions, Sun's Earth's, Moon's, etc.

In my view, the arguments put forward above about beautifying|simplifying the code, don't strike me as particularly urgent at this (difficult) time for Celestia development.

I disagree about the necessity for this sort of utility coding.

In order to better manage desires between fixing bugs versus upgrading code/functionality, may I propose separating out the latter for a notional Celestia version 2.0+ release?

Spiff.