new star.dat

[HB]

Hi,

Past weekend I've downloaded the latest CVS- version of Celestia.
Before compile or use the latest version I am used to check the changes.

What I've seen is that stars.dat is completly changed and two indexfiles are added(SAOxindex.dat an HDxindex.dat)
Of coarse some of the source files has been changed as well to make those *.dat files useable.

Concerning this, my question is why are the spherical coordinates changed into rectangular coordinates??
And why are the HD numbers separated and moved to a crossreference file??

To avoid high phone-costs of obtain info from the interne, I've made a while ago a database program with all relevant information from the stars especially for the positions of the stars which is usefull in combination with a telescoop. This databaseprogram makes use of the info stored into the stars.dat file.
The new stars.dat file makes the use, other then for Celestia, for this database and perhaps other kind of purposes impossible.

I've tried to reverse the way conversion has made from spherical into rectangular coordinates. I'm failed with that. (short of unknown operands like "distance")
So,is it possible to convert the rectangular coordinates back into spherical ones by only using the (x,y,z) information stored in the new stars.dat file?? If yes how to do it??

[selden]

HB,

For the details, you need to read the recent postings by Chris to the developers' mailing list. The archives are at http://sourceforge.net/mailarchive/forum.php?forum_id=9864

Briefly, however, Cel:// URLs used with Runar's "Billow Maidens" Addon revealed problems in the method used to place stars within Celestia's coordinate system: the spherical coordinate system transforms don't produce exactly the same results when Celestia is built using different compilers. Using a database that's in Celestia's rectangular coordinate sytem ensures that the same star positions will be used on all systems.

Since such a change is going to be incompatible anyhow, other improvements are being included, too, like being independant of HIP or HD cataloging.

I don't think the tools for creating the new format are in the CVS archive, but they are available on shatters.net. Please read the Developers' Archive for details.

[HB]

Selden,

Thanks for your quick response.
Reading all about this matter at the developers mailing list it looks if the discussion is still going and so more changes can be expected in the future.

I've took a close look to the astro.cpp file and in my humble opinion it must be possible to reverse the rectangle coordinates back into spherical ones if the "distance" is knowing.
Am I right in this?? And if so, is it possible to have this distance into the stars.dat file??

For the use of my database program in combination with Celestia I've changed the stardump tool a little bit. So now I'm able to dump a textversion from both the new binary files SAOxindex.dat and hdxindex.dat too.
If the future catalog crossref-files have the same structure as the above mentioned ones this modification might be usefull for other people.

[granthutchison]

I've took a close look to the astro.cpp file and in my humble opinion it must be possible to reverse the rectangle coordinates back into spherical ones if the "distance" is knowing.
Am I right in this?? And if so, is it possible to have this distance into the stars.dat file??

The radial distance is easily calculated from the x,y,z coordinates:

d = sqrt(x^2 + y^2 + z^2)

with perhaps a conversion constant if you want to use different units. So it would be a waste of space to include distance in the new stars.dat file as a separate entry.

Grant

[Evil Dr Ganymede]

grant - is the stars.txt file at
http://www.celestiaproject.net/~claurel/celest ... /stars.txt

the text version of the new file? And all the numbers at the start are the HIP numbers, yes?

Does that include all the recently discovered nearby stars? Or do you need the add-ons that you wrote for those too?

[granthutchison]

grant - is the stars.txt file at
http://www.celestiaproject.net/~claurel/celest ... /stars.txt
the text version of the new file?

No. It contains the old spherical coordinates and apparent magnitudes, rather than the new rectangular coordinates and absolute magnitudes.

And all the numbers at the start are the HIP numbers, yes?

Yes.

Does that include all the recently discovered nearby stars?

No ... they have no Hipparcos catalogue numbers.

Or do you need the add-ons that you wrote for those too?

The Nearby Stars add-on should still work with the new format. But of course there will be a much better add-on once Chris has binary stars implemented ... which is partly why this new stars.dat format is being introduced.

Grant

[HB]

In my attempt to get back the ra,dec and distance from only the x,y,z coordinates a problem appears.

I've used the formulas given in the file astro.cpp:

theta=ra/24*pi*2+pi (A)
phi=(dec/90-1)*pi/2 (B)
x=cos(theta)*sin(phi)*distance (C)
y=cos(phi)*distance (D)
z=-sin(theta)*sin(phi)*distance (E)

As reference I use the infotxt for HIP1:

ra=0.00091185
dec=1.08901334
distance=921.37572
x=921.20929
y=16.06575
z=-6.9790912

My simple mind will say that from (D):

cos(phi)=y/distance and so phi=acos(y/distance)

But phi as calculated from formula (B) is not the same as phi calculated from above formula (D)

phi from (D)= 0.0174310866
phi from (B)= 0.01900572399

What am I doing wrong??

[HB]

I wrote:

phi from (D)= 0.0174310866
phi from (B)= 0.01900572399

But it should be:

COS( phi) from (D)= 0.0174310866
COS( phi) from (B)= 0.01900572399

[granthutchison]

I haven't looked at astro.cpp, but it seems like there's a transformation missing in what you've listed.
What's written would transform a set of spherical coordinates into rectangular coordinates in the same coordinate frame. But RA and dec are equatorial coordinates, whereas Celestia's x,y,z are (as far as I recollect) ecliptic. So one set of coordinate axes is inclined at ~23.5 degrees to the other, and you're therefore not comparing like with like when you derive phi from RA,dec and from x,y,z.

Grant

Edit: Oops. See below.

[granthutchison]

Looks like x,y,z must be equatorial coordinates after all.

Your problem seems to be just a simple misprint, since actually y=16.060575 (note the extra zero!) for Hip 1.
That gives acos(phi) = 0.017431081 from B, which is, within the limits of precision, equal to your calculated value from D.

Grant

Edit: Oh, for crying out loud. See below again.

[chris]

Looks like x,y,z must be equatorial coordinates after all.

x, y, and z should be ecliptic--there's a rotation by ~23.5 degrees in the equatorialToCartesian function.

--Chris

[t00fri]

Looks like x,y,z must be equatorial coordinates after all.

x, y, and z should be ecliptic--there's a rotation by ~23.5 degrees in the equatorialToCartesian function.

--Chris

I seriously doubt that the change of the star data to a rectangular basis is in any direct way related to an increase in (numerical) accuracy. Every physicist's "golden law" says that data should always be input in the same basis they were measured! Given that "natural" basis, one then always retains the best accuracy and /transparency/ in data administration (!) by applying Gauss' error propagation from there. The issue of /numerical/ inaccuracies (extinction errors) can always be taken care of by cleverly regrouping the math operations involved in the computer. That crucial issue is finally responsible for the final numerical accuracy! It is for sure not a matter of the chosen coordinate system for the data...

I have followed of course the various discussions in the developer's list, but kept out of them, since the "rectangular" transformation code was written already /before/ any discussion could have started;-)

Bye Fridger

[granthutchison]

x, y, and z should be ecliptic--there's a rotation by ~23.5 degrees in the equatorialToCartesian function.

I'm just going daft, don't worry. It's probably time for my lie-down.
The misprint just fine-tunes the difference, rather than resolving it ... I got my equation B and equation D mixed up, and ended up comparing one result with itself, rather than with the other. Idiot.
So, yes, as you were, the difference really is because the coordinates axes are different.

Grant

[chris]

Looks like x,y,z must be equatorial coordinates after all.

x, y, and z should be ecliptic--there's a rotation by ~23.5 degrees in the equatorialToCartesian function.

--Chris

I seriously doubt that the change of the star data to a rectangular basis is in any direct way related to an increase in (numerical) accuracy.

The issue is not one of accuracy but of consistency. There were single bit differences in the spherical to rectangular conversion on different systems. Since at some point such a conversion must be performed (OpenGL deals in rectangular, not spherical coordinates), why not do it when the star database is built, thereby ruling out potential differences from the compiler or CPU?

Every physicist's "golden law" says that data should always be input in the same basis they were measured! Given that "natural" basis, one then always retains the best accuracy and /transparency/ in data administration (!) by applying Gauss' error propagation from there. The issue of /numerical/ inaccuracies (extinction errors) can always be taken care of by cleverly regrouping the math operations involved in the computer.

Right, but the compiler is ultimately responsible for the ordering of math operations, and it's already been demonstrated by the cel:// URL bug that different compilers generated different code for the exact same expression.

I have followed of course the various discussions in the developer's list, but kept out of them, since the "rectangular" transformation code was written already /before/ any discussion could have started;-)

It's not too late to switch back to using spherical coordinates if you can provide a compelling reason to do so.

--Chris

[granthutchison]

OK, by way of an apology for the confusion I created, here's how to find your way back to RA and dec from x, y and z:

1) Calculate ecliptic longitude (lambda) and latitude (beta):

lambda = atan(-z/x)

beta = atan(y/sqrt(x^2+z^2))

2) Convert ecliptic coordinates to RA (alpha) and declination (delta), using the J2000.0 obliquity of the ecliptic (epsilon) = 23.4392911 deg

alpha = atan{[sin(lambda)*cos(epsilon) - tan(beta)*sin(epsilon)]/cos(lambda)}

delta = asin{sin(beta)*cos(epsilon) + cos(beta)*sin(epsilon)*sin(lambda)}

(Remember to keep track of the correct quadrant for all those atan functions ...)

Grant

[chris]

(Remember to keep track of the correct quadrant for all those atan functions ...)

Or just let atan2 do it for you:

lambda = atan2(-z, x)

Most math libraries should have this function.

--Chris

[granthutchison]

The issue is not one of accuracy but of consistency.

Assuming we do move to rectangular coordinates for stars.dat ... for the sake of consistency, would it then be appropriate to introduce the option of x,y,z parameters in stc files?
I never really understood why matched spherical coordinates in stars.dat and an stc didn't position stars in exactly the same place in a given compiled Celestia. (Although perhaps in a different same place from someone else's Celestia, if you see what I mean.) But if we're mixing rectangular coords in stars.dat with spherical in the stc, then presumably we're going to end up with offsets between stars.dat stars and add-on stars which might vary from user to user.
Or am I having another delusional episode?

Grant

[chris]

The issue is not one of accuracy but of consistency.

Assuming we do move to rectangular coordinates for stars.dat ... for the sake of consistency, would it then be appropriate to introduce the option of x,y,z parameters in stc files?

This is something I plan on adding soon. It will be an option, because often it's much more convenient to use spherical coordinates.

I never really understood why matched spherical coordinates in stars.dat and an stc didn't position stars in exactly the same place in a given compiled Celestia.

I think that the difference was due to slight to small differences in the use of double precision intermediates in the two conversions.

--Chris

[HB]

Anyway, thank you very much to all of you so far.

Hans.

[t00fri]

I seriously doubt that the change of the star data to a rectangular basis is in any direct way related to an increase in (numerical) accuracy.

The issue is not one of accuracy but of consistency. There were single bit differences in the spherical to rectangular conversion on different systems.
...

--Chris

Chris,

after having struggled for decades with the general problematic of retaining the original precision of "data" in extensive numerical calculations involving those data, I consider your above "xyz" approach towards "consistency" both ill-defined and really not to the point.

-- numerical consistency certainly does NOT imply that some numerically calculated function of the input "data" should be identical on different machine architectures, notably if calculated with different compilers!

As you pointed out already, modern compilers along with different optimization algorithms take a lot of freedom in performing the actual numerical evaluations.

-- After making the result look identical among different compilers and machines (using your external x,y,z conversion), it could well happen that it is nevertheless numerically /inaccurate/ everywhere;-). This is for sure not a consistent solution...

------------------------------
A sensible i.e. satisfactory requirement of consistency rather is that the numerical math errors (extinction, rounding...) merely have to remain smaller than the Gauss propagated physical errors. With different compilers, on different machines...
------------------------------

Nothing else is really relevant.

Please note that in the above "consistency criterion", I compare the true i.e. "physical" uncertainties associated with the 2d star position or parallax measurements with the numerical math errors (extinction, rounding,...).


Everyone who has heard or taught(;-)) respective courses on numerical math knows how to achieve the above aim of consistency straightforwardly.

The propagation of relative physical errors across the chain of numerical calculations can be checked by varying the initial star data slightly and tracking the effects of the subsequent manipulations via Gauss error propagation. A routine task. Depending on the transformations involved there may be drastic surprises here: It frequently happens that a small change in the original data (within their given uncertainties) will induce huge variations in the final result. Such a situation is clearly bad and should be avoided if at all possible...

I claim that the significant inaccuracies viz "inconsistencies" people have observed in star positions can be taken care of by rearranging the analytic conversion such that physical precision remains uniformly the same.

Carrying along the star data in this unintuitive x,y,z representation in my view is really not well motivated in view of the above considerations...

Bye Fridger