New Celestia unified runtime database

Post #41by **Lafuente_Astronomy** » 19.04.2019, 22:18

That's alright. More work, if done with th intention of making an excellent finished product, is worth it right? I'd rather have a long-released and great quality Celestia program than a rushed one

Post #42by **Lafuente_Astronomy** » 24.04.2019, 00:30

Hello Celestians! How are the developments going?

Post #43by **pirogronian** » 24.04.2019, 02:50

Lafuente_Astronomy wrote:Hello Celestians! How are the developments going?

Slow but steady. New database is in half ready but integration with existing code is a task for a separate thread... :razz:

Post #44by **Lafuente_Astronomy** » 24.04.2019, 06:27

Oh. That's good to hear! Slow and steady results in a much better completed work than a fast and rushed one. But what do you specifically mean by "Integration with existing code"?

Added after 2 minutes 4 seconds:

pirogronian wrote:integration with existing code is a task for a separate thread...

I think this thread can be fine for another development task. But since you're one of the developers, then do what you think is best :smile:

Post #45by **pirogronian** » 24.04.2019, 07:32

New database has radically different structure, so has also slightly different api. Sadly, some legacy code is tightly connected with render code. It already compiles successuly, but behaviour after integration is still to be tested.

Post #46by **Lafuente_Astronomy** » 24.04.2019, 09:05

Ohhh. I know that by the end of this, Celestia will be much better than ever before!

Post #47by **pirogronian** » 26.04.2019, 20:22

@Lafuente_Astronomy

The good news and some not so good news. The good one - integration of first phase of new database is done and working!
Not so good - fps dropped twice, from 60 (probably forced by vsync) to merely 30... Looking for delaying code will be worse than looking for bugs :rambo:

Post #48by **Lafuente_Astronomy** » 26.04.2019, 21:12

Oh dear. Perhaps introducing something new but stable will solve that problem perhaps?

Janus · Post #49by **Janus** » 26.04.2019, 23:12

@pirogronian

Will the new code be available for others to compile and test anytime soon?
Maybe as a branch on github like double is?

Currently I am stuck rebuilding my primary work machine after making another attempt at VS2017 on my Win7.
Made it further, but I also couldn't save it this time. I so hate the garbage M$ is calling a GUI, still, it is better than their latest piece of fecal matter.
I will have everything up in the next day or two, and a distraction would be nice.

Waiting on a subcontractor of current customer to explain why a driver has a binary blob in it that doesn't show up in the sources.
Took a couple of weeks to reverse the glitching driver, correlate code to source, and I had left overs.
Currently on paid standby to review the subcontractor's new code after they fix their "oops" of sending the wrong assembly.

Janus.

Post #50by **pirogronian** » 27.04.2019, 02:25

@Janus
It already is :smile:

https://github.com/CelestiaProject/Celestia/tree/astrodb?files=1

Post #51by **Lafuente_Astronomy** » 27.04.2019, 08:33

In relation to having the code already placed, and since one of the steps to improving Celestia 1.7.0 is to improve Celestia's databases with updated data from all the astronomical catalogs and surveys, especially the latest and most comprehensive and detailed catalogs and surveys, does anyone here know how to obtain Gaia Data and convert it for use in Celestia? I'll link you the official website of the Gaia Archive so that it'll be easier for you to access.

https://gea.esac.esa.int/archive/

Post #52by **pirogronian** » 27.04.2019, 09:15

@Lafuente_Astronomy
I want to solve performace issue first and be sure that this code will be accepted and will be merged into master branch.

Post #53by **Lafuente_Astronomy** » 27.04.2019, 09:17

I agree. Optimization and fixing of the code takes priority. Otherwise, it won't be able to function as planned and designed.

Janus · Post #54by **Janus** » 28.04.2019, 16:29

@pirogronian

Referring to the astrodb branch.
Am I reading this correctly?
Did you put stars & DSOs into the same database?

Been trying to get it to compile via VS only, and failed.
Is there a plan for dealing with code intended to deal with stars only?

Janus.

Post #55by **pirogronian** » 28.04.2019, 19:43

Janus wrote:Am I reading this correctly?
Did you put stars & DSOs into the same database?

Yes, and bodies are also to be put there. All astronomical objects will be in one unified database.

I work on Linux, so Windows related code is updated with delay - I have to check Appveyor log for compile errors, so it takes more time.

My overall goal is to unify rather than splitting. So no plan for star only code. Database will be for all astronomical objects via astronomical catalogs. Maybe some small exceptional pieces here and there, if will be really needed.

Janus · Post #56by **Janus** » 28.04.2019, 20:12

Will there be an index for stars only?
How are you going to handle putting stars and planets, which have very different data needs, into the same database?
Are you planning a per item structure union/merge for everything?

Please indicate there will be indexes for different types of things.
With out those indexes, every single item will have to be searched to find anything.
It sounds like a performance hog to me.
However, I am not a real C/C++ programmer, so I could be wrong.

In the mean time I will stick to playing with the main branch.
Once I realized they were merged, so there was no way to do a find of just stars, I stopped.

Janus.

Post #57by **pirogronian** » 28.04.2019, 22:24

Main goal of database is searching, so I hope there will be no performance hog. Iterating over all stars is possible via getStars() which returns reference to set container. Analogously for DSOs. Main index is shared, because it is for cross index referencing only.

I merged some common code (currently categories support and main index, but I plan make all movable like bodies and stars). Good example of common interface for various kinds of objects is Selection class. Actually I plan to reimplement most of it.

Post #58by **onetwothree** » 30.04.2019, 09:31

I'd suggest instead of reinventing the wheel (square wheel

) looking through OpenSpace's and GayaSky's implementations of star databases. Especially the latest one as it can handle billions of stars.

Post #59by **Lafuente_Astronomy** » 30.04.2019, 11:27

onetwothree wrote:I'd suggest instead of reinventing the wheel (square wheel ) looking through OpenSpace's and GayaSky's implementations of star databases. Especially the latest one as it can handle billions of stars.

What does that mean, to be specific? Like several star.dat files with each file based on differing catalogs or a single massive star.dat file which allows multiple object identifiers to be utilized, especially Celestia's personal universal object identifier?

Post #60by **onetwothree** » 30.04.2019, 12:09

Dunno. I haven't checked them actually