Updated stars.dat including Gaia EDR3 data (update 2022-03-03)

[DrJMOS]

Hi Ajtribick

You will see a result with 1 row. There is a small icon which looks like a table, which shows a tooltip "Display top 2000 results". Click it.
Take a look at the value in the r_est column, which contains the estimated distance in parsecs.

...is done

mean 1.3737 kpc is true?

Ajtribick, do you know a script to mark all O-B type stars of your addon too?

Added after 4 hours 50 minutes:
I found Kaus Media star distance to 4346 parsec and Garnet Star at 3070 parsec in Gaia DR2 but i can not find in Gaia DR2 to the next stars:

Aludra
Aspidiske
HD 105435
30 CMa

[ajtribick]

mean 1.3737 kpc is true?

I see you didn't check what was inside the spoiler tag...
Gaia cannot measure the brightest stars, and of the ones it can measure the bright ones tend to have fairly unreliable parallaxes. Since at the moment I only take the XHIP distances if there is no Gaia parallax, stars like Alpha Cassiopeiae and Delta Sagittarii end up being placed at distances computed from the unreliable Gaia parallax.

Ajtribick, do you know a script to mark all O-B type stars of your addon too?

No, but this should be fairly straightforward to implement. I leave it as an exercise for the reader

[DrJMOS]

No, but this should be fairly straightforward to implement. I leave it as an exercise for the reader

Yep! very helpful to fix arms of the Milky Way viewtopic.php?f=5&t=20576

[ajtribick]

I've released a new version v1.0.4 which uses relative parallax error to choose between Gaia DR2 and XHIP parallaxes (this affects the cases of Alpha Cas and Delta Sgr mentioned by DrJMOS, they should be back at more reasonable distances). This affects 1787 stars out of a total 2499574. There's also some refactoring of the code involved but that's not particularly relevant for actually using the data files.

See the top of the thread for the link.

(This is still compatible with pre-1.6.2 Celestia)

[DrJMOS]

I've released a new version v1.0.4 which uses relative parallax error to choose between Gaia DR2 and XHIP parallaxes (this affects the cases of Alpha Cas and Delta Sgr mentioned by DrJMOS, they should be back at more reasonable distances). This affects 1787 stars out of a total 2499574. There's also some refactoring of the code involved but that's not particularly relevant for actually using the data files..

Hi Ajtribick,

Thank you, great fix, works better!!!

[Art Blos]

DrJMOS < What are these red marks in the screenshot?

[DrJMOS]

DrJMOS < What are these red marks in the screenshot?

Hi Art Blos, red marks is a custom adaptation of the original addon Graticules of Selden Ball to see distances inside the Milky way in light years in galactic plane (graticule of 1 ly to 200000 ly). Beyond the galaxy continue the distances with council of giants ring (yellow circle) and Redshift distances z=0.01 to z=1000 (in red circles onto edge of Planck background)

https://www.classe.cornell.edu/~seb/celestia/graticules.html#4

[Art Blos]

Hi Art Blos, red marks is a custom adaptation of the original addon Graticules of Selden Ball to see distances inside the Milky way in light years in galactic plane (graticule of 1 ly to 200000 ly). Beyond the galaxy continue the distances with council of giants ring (yellow circle) and Redshift distances z=0.01 to z=1000 (in red circles onto edge of Planck background)

Can you share your version?

[DrJMOS]

Can you share your version?

of course, I'll create a new post: viewtopic.php?f=23&t=20629

[ajtribick]

A quick update with regards to the now-released Gaia EDR3.

The process I'm using to generate stars.dat relies on several sources external to the Gaia release itself, including:

• The geometric distance catalogue, that estimates the distances with an exponentially-decaying distance prior based on a galactic model rather than a simple 1/parallax calculation.
• More complete cross-matches to HIP2/TYC2 than the cross-match tables included in the Gaia releases. A quick check of the number of entries in the gaiaedr3.hipparcos2_best_neighbour table shows it is more complete than DR2, but it still falls short of the source I'm actually using for the cross-match.

Equivalent sources for Gaia EDR3 do not exist yet, so I'm not going to be creating an EDR3-based stars.dat in the immediate future.

[ajtribick]

An alpha build of the EDR3-based stars.dat, using my own cross-matching code for HIP-Gaia and TYC-Gaia, I give no guarantees on correctness.

Still a few more features I want to add before I consider it done, and I haven't done too much verification other than checking a few test stars have ended up in the right constellations. The DR2 database is still probably a safer bet, especially with regards to the HIP-Gaia crossmatch.

Still got a few more items on the to-do list, including HIP1 stars, the part of TYC2 supplement 1 that's not in TDSC, and an attempt at eliminating the Magellanic spikes.

Future updates will use the following version scheme:

Major version = stars.dat format (current format = 1)
Minor version = Gaia data release: 0 = DR2, 1 = EDR3, 2 = DR3
Patch version = incremental bugfixes

https://github.com/ajtribick/celestia-gaia-stardb/releases/tag/v1.1.0-alpha.1

[Sirius_Alpha]

I just downloaded and have spent some time exploring A. J. Tribick's new Gaia EDR3-based stars.dat.

With better parallaxes, clusters don't look as stretched away from the sun. Praesepe looks great! It looked good in DR2 but it looks much less distorted in EDR3. As another more random example, NGC 3532 looks far better in this new stars.dat. In the DR2-based stars.dat, NGC 3532 was kind of a long string. In this new stars.dat file, NGC 3532 (while definitely stretched) is far more recognizable as a cluster. Cranking up the magnitude limit and stepping back away from the solar neighborhood, many more clusters are easily visible than before.

I know it's still a work in progress but I'm very happy with it as it is so far, and I really appreciate you putting all that effort into it, Tribick!

[Eric Nelson]

There's a lot to map out and EDR-3 is something for sure.
There's a lot that's been observed and mapped and each data release did it in individual quality.
Though GAIA in general has too much astrometric noise in the observations (making it practically unreliable) and its successors should be a big improvement in detection and observation of what's out there and what such is like.
Imagine what a successor to GAIA and any of its data releases would be like in the coming years and decades.
Imagine what other things like James Webb (despite the delays) and better telescopes would do for observations.
They would also have to map out individual things including the stars that were measured and then we can get some better analysis about them than we ever got along with better mapping.
We can only go so far with our knowledge.
Guys worked very hard on these Celestia star updates and work though.

[ajtribick]

A small Python utility to extract the data from stars.dat and dump it into a CSV file (actually comma-separated, so be careful if your regional settings make Excel assume semicolons as separators). Should work with Python 3.6+, and only requires the Python standard library.

The stars.dat file is assumed to be in the current directory, and the result is written to stdout. The x, y, z columns represent the coordinates as stored in the file, while RA, Dec, Dist are computed (and may not round-trip due to limitations in precision and differences in the implementation of trigonometric functions).

This is provided as-is, no guarantees that it will be useful. In particular, the size of the CSV from running against the stars.dat files in this thread is probably too large to handle in typical spreadsheet software.

[ajtribick]

Here's a second alpha version. The distances for non-crossmatched Hipparcos stars (mostly stars that are too bright for Gaia) are now estimated using the same method used to produce the geometric distances. This method uses Monte Carlo methods which require random numbers so I've included the generated file in the source for reproducibility. There was also an instance where the alpha.1 code used magnitude transformation formulae from DR2, this has now been updated to the DR3 version. In practice, the changes should be fairly minor.

I'm now also outputting auxiliary files, currently these are the estimated distances in parsecs from HIP2 parallaxes (median and 68% confidence limits) and the generated cross-matches: at the moment the cross-matches are just the IDs of the best match, I may decide to add more details in later releases.

https://github.com/ajtribick/celestia-gaia-stardb/releases/tag/v1.1.0-alpha.2

[ajtribick]

Updated the cross-matching from HIP/TYC to Gaia to take into account the HIP ids in the TYC catalogue. I'm calling this one a beta. I've temporarily removed the auxiliary file generation, will re-enable it for the final release.

https://github.com/ajtribick/celestia-gaia-stardb/releases/tag/v1.1.0-beta.1

[ajtribick]

Made a release version based on Gaia EDR3, managed to do it before the full DR3 comes out. Details in the first post in the thread.

There are still additional improvements that could be made, e.g. trying to remove the Magellanic spikes, but I'm leaving these as "for future consideration". Depending on how much of the code I need to revise for DR3, I might do it then.

[ajtribick]

New alpha build, with an updated scoring function for the cross-match which incorporates parallaxes and fixes an issue where the highest weight was effectively given to the parameter (e.g. proper motion, magnitude difference) with the worst matches.

Checking the distances of a few of the stars with differences, HIP 27, 47, 65, 111, 352, 15973, 92916 now seem to be more in-line with SIMBAD. The cross-match for HIP 8716 gives a worse distance in the new version.

Designating this as an alpha as I will likely do some further tweaking, plus I am probably going to expand the magnitude difference cut-off to avoid some correct matches being eliminated at early stages.

https://github.com/ajtribick/celestia-gaia-stardb/releases/tag/v1.1.1-alpha.1

[DrJMOS]

Hello AjTribick, i love your star catalogue and the add of the extrasolar planets of Sirius Alpha, but when I've tried to install both adds appear conflicts, for example, gliese 667 appear with 4 suns. My computer is slow and I can't manipulate the 40mb file to remove duplicate stars, can you create a version without the stars that appear in the Sirius Alpha exoplanet catalogue?

[SevenSpheres]

It looks like the problem is that the old version of nearstars.stc that comes with Celestia 1.6.x is missing TYC designations for Gliese 667 A and B, which will result in duplicates with the updated star database. This has been fixed in the current version of nearstars.stc, so just replace it.

See here for a guide to keeping Celestia's default content up to date.