Updated stars.dat including Gaia EDR3 data (update 2022-03-03)

The home for finished/released addons.
Avatar
DrJMOS
Posts: 101
Joined: 08.08.2020
Age: 42
With us: 3 years 10 months
Contact:

Post #61by DrJMOS » 29.11.2020, 01:13

Hi Ajtribick

You will see a result with 1 row. There is a small icon which looks like a table, which shows a tooltip "Display top 2000 results". Click it.
Take a look at the value in the r_est column, which contains the estimated distance in parsecs.

ventana2.png

ventana1.png


...is done

mean 1.3737 kpc is true? :biggrin:

Ajtribick, do you know a script to mark all O-B type stars of your addon too?

Added after 4 hours 50 minutes:
I found Kaus Media star distance to 4346 parsec and Garnet Star at 3070 parsec in Gaia DR2 but i can not find in Gaia DR2 to the next stars:

Aludra
Aspidiske
HD 105435
30 CMa
My avatar is part of my custom splash viewtopic.php?f=5&t=20457#p151900
List of my addons here viewtopic.php?f=23&t=20724

Topic author
ajtribick
Developer
Posts: 1855
Joined: 11.08.2003
With us: 20 years 10 months

Post #62by ajtribick » 29.11.2020, 13:22

DrJMOS wrote:mean 1.3737 kpc is true?
I see you didn't check what was inside the spoiler tag... :)
Gaia cannot measure the brightest stars, and of the ones it can measure the bright ones tend to have fairly unreliable parallaxes. Since at the moment I only take the XHIP distances if there is no Gaia parallax, stars like Alpha Cassiopeiae and Delta Sagittarii end up being placed at distances computed from the unreliable Gaia parallax.

DrJMOS wrote:Ajtribick, do you know a script to mark all O-B type stars of your addon too?
No, but this should be fairly straightforward to implement. I leave it as an exercise for the reader :)

Avatar
DrJMOS
Posts: 101
Joined: 08.08.2020
Age: 42
With us: 3 years 10 months
Contact:

Post #63by DrJMOS » 29.11.2020, 15:07

No, but this should be fairly straightforward to implement. I leave it as an exercise for the reader :)

Yep! very helpful to fix arms of the Milky Way viewtopic.php?f=5&t=20576
My avatar is part of my custom splash viewtopic.php?f=5&t=20457#p151900
List of my addons here viewtopic.php?f=23&t=20724

Topic author
ajtribick
Developer
Posts: 1855
Joined: 11.08.2003
With us: 20 years 10 months

Post #64by ajtribick » 30.11.2020, 19:33

I've released a new version v1.0.4 which uses relative parallax error to choose between Gaia DR2 and XHIP parallaxes (this affects the cases of Alpha Cas and Delta Sgr mentioned by DrJMOS, they should be back at more reasonable distances). This affects 1787 stars out of a total 2499574. There's also some refactoring of the code involved but that's not particularly relevant for actually using the data files.

See the top of the thread for the link.

(This is still compatible with pre-1.6.2 Celestia)

Avatar
DrJMOS
Posts: 101
Joined: 08.08.2020
Age: 42
With us: 3 years 10 months
Contact:

Post #65by DrJMOS » 02.12.2020, 07:38

I've released a new version v1.0.4 which uses relative parallax error to choose between Gaia DR2 and XHIP parallaxes (this affects the cases of Alpha Cas and Delta Sgr mentioned by DrJMOS, they should be back at more reasonable distances). This affects 1787 stars out of a total 2499574. There's also some refactoring of the code involved but that's not particularly relevant for actually using the data files..

Hi Ajtribick,

Thank you, great fix, works better!!! :clap: :clap: :smoke:

milky way.jpg
My avatar is part of my custom splash viewtopic.php?f=5&t=20457#p151900
List of my addons here viewtopic.php?f=23&t=20724

Avatar
Art Blos M
Moderator
Posts: 1129
Joined: 31.08.2017
Age: 31
With us: 6 years 9 months
Location: Volgodonsk, Rostov Oblast, Russia

Post #66by Art Blos » 02.12.2020, 07:54

DrJMOS < What are these red marks in the screenshot?
Founder and head of the project "Celestia Origin"

Avatar
DrJMOS
Posts: 101
Joined: 08.08.2020
Age: 42
With us: 3 years 10 months
Contact:

Post #67by DrJMOS » 02.12.2020, 08:12

DrJMOS < What are these red marks in the screenshot?

Hi Art Blos, red marks is a custom adaptation of the original addon Graticules of Selden Ball to see distances inside the Milky way in light years in galactic plane (graticule of 1 ly to 200000 ly). Beyond the galaxy continue the distances with council of giants ring (yellow circle) and Redshift distances z=0.01 to z=1000 (in red circles onto edge of Planck background)

https://www.classe.cornell.edu/~seb/celestia/graticules.html#4

galaxiaen rojo.jpg

conciliouss.jpg

universo.jpg
My avatar is part of my custom splash viewtopic.php?f=5&t=20457#p151900
List of my addons here viewtopic.php?f=23&t=20724

Avatar
Art Blos M
Moderator
Posts: 1129
Joined: 31.08.2017
Age: 31
With us: 6 years 9 months
Location: Volgodonsk, Rostov Oblast, Russia

Post #68by Art Blos » 02.12.2020, 08:58

DrJMOS wrote:Hi Art Blos, red marks is a custom adaptation of the original addon Graticules of Selden Ball to see distances inside the Milky way in light years in galactic plane (graticule of 1 ly to 200000 ly). Beyond the galaxy continue the distances with council of giants ring (yellow circle) and Redshift distances z=0.01 to z=1000 (in red circles onto edge of Planck background)
Can you share your version?
Founder and head of the project "Celestia Origin"

Avatar
DrJMOS
Posts: 101
Joined: 08.08.2020
Age: 42
With us: 3 years 10 months
Contact:

Post #69by DrJMOS » 03.12.2020, 01:08

Art Blos wrote:Can you share your version?
of course, I'll create a new post: viewtopic.php?f=23&t=20629
My avatar is part of my custom splash viewtopic.php?f=5&t=20457#p151900
List of my addons here viewtopic.php?f=23&t=20724

Topic author
ajtribick
Developer
Posts: 1855
Joined: 11.08.2003
With us: 20 years 10 months

Post #70by ajtribick » 03.12.2020, 22:12

A quick update with regards to the now-released Gaia EDR3.

The process I'm using to generate stars.dat relies on several sources external to the Gaia release itself, including:

  • The geometric distance catalogue, that estimates the distances with an exponentially-decaying distance prior based on a galactic model rather than a simple 1/parallax calculation.
  • More complete cross-matches to HIP2/TYC2 than the cross-match tables included in the Gaia releases. A quick check of the number of entries in the gaiaedr3.hipparcos2_best_neighbour table shows it is more complete than DR2, but it still falls short of the source I'm actually using for the cross-match.

Equivalent sources for Gaia EDR3 do not exist yet, so I'm not going to be creating an EDR3-based stars.dat in the immediate future.

Topic author
ajtribick
Developer
Posts: 1855
Joined: 11.08.2003
With us: 20 years 10 months

Post #71by ajtribick » 26.06.2021, 19:38

An alpha build of the EDR3-based stars.dat, using my own cross-matching code for HIP-Gaia and TYC-Gaia, I give no guarantees on correctness.

Still a few more features I want to add before I consider it done, and I haven't done too much verification other than checking a few test stars have ended up in the right constellations. The DR2 database is still probably a safer bet, especially with regards to the HIP-Gaia crossmatch.

Still got a few more items on the to-do list, including HIP1 stars, the part of TYC2 supplement 1 that's not in TDSC, and an attempt at eliminating the Magellanic spikes.

Future updates will use the following version scheme:

Major version = stars.dat format (current format = 1)
Minor version = Gaia data release: 0 = DR2, 1 = EDR3, 2 = DR3
Patch version = incremental bugfixes

https://github.com/ajtribick/celestia-gaia-stardb/releases/tag/v1.1.0-alpha.1

Avatar
Sirius_Alpha
Posts: 214
Joined: 21.03.2019
With us: 5 years 2 months

Post #72by Sirius_Alpha » 26.06.2021, 21:20

I just downloaded and have spent some time exploring A. J. Tribick's new Gaia EDR3-based stars.dat.

With better parallaxes, clusters don't look as stretched away from the sun. Praesepe looks great! It looked good in DR2 but it looks much less distorted in EDR3. As another more random example, NGC 3532 looks far better in this new stars.dat. In the DR2-based stars.dat, NGC 3532 was kind of a long string. In this new stars.dat file, NGC 3532 (while definitely stretched) is far more recognizable as a cluster. Cranking up the magnitude limit and stepping back away from the solar neighborhood, many more clusters are easily visible than before.

I know it's still a work in progress but I'm very happy with it as it is so far, and I really appreciate you putting all that effort into it, Tribick!
Exoplanet nerd. I maintain a monthly-updated exoplanet catalogue here:
https://celestiaproject.space/forum/viewtopic.php?f=23&t=18705

Eric Nelson
Posts: 119
Joined: 03.06.2021
With us: 3 years

Post #73by Eric Nelson » 29.06.2021, 01:03

There's a lot to map out and EDR-3 is something for sure.
There's a lot that's been observed and mapped and each data release did it in individual quality.
Though GAIA in general has too much astrometric noise in the observations (making it practically unreliable) and its successors should be a big improvement in detection and observation of what's out there and what such is like.
Imagine what a successor to GAIA and any of its data releases would be like in the coming years and decades.
Imagine what other things like James Webb (despite the delays) and better telescopes would do for observations.
They would also have to map out individual things including the stars that were measured and then we can get some better analysis about them than we ever got along with better mapping.
We can only go so far with our knowledge.
Guys worked very hard on these Celestia star updates and work though.

Topic author
ajtribick
Developer
Posts: 1855
Joined: 11.08.2003
With us: 20 years 10 months

Post #74by ajtribick » 20.07.2021, 17:03

A small Python utility to extract the data from stars.dat and dump it into a CSV file (actually comma-separated, so be careful if your regional settings make Excel assume semicolons as separators). Should work with Python 3.6+, and only requires the Python standard library.

The stars.dat file is assumed to be in the current directory, and the result is written to stdout. The x, y, z columns represent the coordinates as stored in the file, while RA, Dec, Dist are computed (and may not round-trip due to limitations in precision and differences in the implementation of trigonometric functions).

This is provided as-is, no guarantees that it will be useful. In particular, the size of the CSV from running against the stars.dat files in this thread is probably too large to handle in typical spreadsheet software.
Attachments
stardb.zip
(1.11 KiB) Downloaded 291 times

Topic author
ajtribick
Developer
Posts: 1855
Joined: 11.08.2003
With us: 20 years 10 months

Post #75by ajtribick » 05.08.2021, 08:28

Here's a second alpha version. The distances for non-crossmatched Hipparcos stars (mostly stars that are too bright for Gaia) are now estimated using the same method used to produce the geometric distances. This method uses Monte Carlo methods which require random numbers so I've included the generated file in the source for reproducibility. There was also an instance where the alpha.1 code used magnitude transformation formulae from DR2, this has now been updated to the DR3 version. In practice, the changes should be fairly minor.

I'm now also outputting auxiliary files, currently these are the estimated distances in parsecs from HIP2 parallaxes (median and 68% confidence limits) and the generated cross-matches: at the moment the cross-matches are just the IDs of the best match, I may decide to add more details in later releases.

https://github.com/ajtribick/celestia-gaia-stardb/releases/tag/v1.1.0-alpha.2

Topic author
ajtribick
Developer
Posts: 1855
Joined: 11.08.2003
With us: 20 years 10 months

Post #76by ajtribick » 09.10.2021, 19:38

Updated the cross-matching from HIP/TYC to Gaia to take into account the HIP ids in the TYC catalogue. I'm calling this one a beta. I've temporarily removed the auxiliary file generation, will re-enable it for the final release.

https://github.com/ajtribick/celestia-gaia-stardb/releases/tag/v1.1.0-beta.1

Topic author
ajtribick
Developer
Posts: 1855
Joined: 11.08.2003
With us: 20 years 10 months

Post #77by ajtribick » 20.12.2021, 16:47

Made a release version based on Gaia EDR3, managed to do it before the full DR3 comes out. :) Details in the first post in the thread.

There are still additional improvements that could be made, e.g. trying to remove the Magellanic spikes, but I'm leaving these as "for future consideration". Depending on how much of the code I need to revise for DR3, I might do it then.

Topic author
ajtribick
Developer
Posts: 1855
Joined: 11.08.2003
With us: 20 years 10 months

Post #78by ajtribick » 15.01.2022, 21:21

New alpha build, with an updated scoring function for the cross-match which incorporates parallaxes and fixes an issue where the highest weight was effectively given to the parameter (e.g. proper motion, magnitude difference) with the worst matches.

Checking the distances of a few of the stars with differences, HIP 27, 47, 65, 111, 352, 15973, 92916 now seem to be more in-line with SIMBAD. The cross-match for HIP 8716 gives a worse distance in the new version.

Designating this as an alpha as I will likely do some further tweaking, plus I am probably going to expand the magnitude difference cut-off to avoid some correct matches being eliminated at early stages.

https://github.com/ajtribick/celestia-gaia-stardb/releases/tag/v1.1.1-alpha.1

Avatar
DrJMOS
Posts: 101
Joined: 08.08.2020
Age: 42
With us: 3 years 10 months
Contact:

Extrasolar Planets (updated catalogue)

Post #79by DrJMOS » 06.02.2022, 19:18

Hello AjTribick, i love your star catalogue and the add of the extrasolar planets of Sirius Alpha, but when I've tried to install both adds appear conflicts, for example, gliese 667 appear with 4 suns. My computer is slow and I can't manipulate the 40mb file to remove duplicate stars, can you create a version without the stars that appear in the Sirius Alpha exoplanet catalogue?
My avatar is part of my custom splash viewtopic.php?f=5&t=20457#p151900
List of my addons here viewtopic.php?f=23&t=20724

Avatar
SevenSpheres
Moderator
Posts: 823
Joined: 08.10.2019
With us: 4 years 8 months

Post #80by SevenSpheres » 06.02.2022, 19:56

It looks like the problem is that the old version of nearstars.stc that comes with Celestia 1.6.x is missing TYC designations for Gliese 667 A and B, which will result in duplicates with the updated star database. This has been fixed in the current version of nearstars.stc, so just replace it.

See here for a guide to keeping Celestia's default content up to date.
My Addons: viewtopic.php?f=23&t=19978 • Discord server admin
Celestia versions: 1.5.1, 1.6.1, 1.6.2, 1.7.0, and some unofficial versions like Celestia-ED


Return to “Add-on releases”