How can Celestia handle 20GB Gaia star data?
Posted: 11.08.2016, 07:23
There were an interesting topic on the Space Engine forum (I copied it down below). It is about handling very large data in Space Engine.
Is it possible to develop something similar in Celestia?
How much work is it?
Can it be borrowed from Space Engine?
***
http://en.spaceengine.org/forum/8-332-1
Transfer data to user can be made like transfer of textures are in Google Earth. The database is organized in octree, and this allows it to handle hundreds of billions stars and the interactive choosing of visible ones, like I've done in SpaceEngine.
The galaxy is subdivided into 1000 pc blocks. Each is subdivided into 8 childs 500 pc each. Each of them is subdivided into 8 childs 250 pc each and so on. SpaceEngine has 10-levels of octree. Each octree block has a star array associated with it. The magic is in the range of absolute magnitude for each octree level. The level 0 (1000 pc blocks) handle stars brighter than -5m abs mag, the level 1 handle -5...0m stars, the level 2 handle 0...+2.5m and so on. The step in magnitude is chosen so that each block contains approximately the same number of stars.
Render (or query for downloading) is based on limiting visual magnitude. For example, limiting visual mag = +6m. The Engine passes the octree and chooses the blocks that have visual magnitude of the most brightest block's star is less than +6m. At limiting magnitude +6m and infinite culling radius (sky render mode), the engine needs ~6000 stars in the Sun neighboorhood, at +7m ~50000 stars. This is just several tens of blocks, these will be downloaded (or generated) very quickly. The octree with magnitude culling allows avoiding querying of distant dwarf stars, that are already invisible. Once the blocks are downloaded from server, they may be saved in the disk cache. (For procedural stars, SpaceEngine doesn't use a disk cache, because generating of blocks are much faster than downloading them from disk.) Of course this system makes no sense for HIPPARCOS catalog with only 120k star, but for Gaia it will be neccessary (however, Gaia will be released in 2018-2019, our computers may have terrabytes of RAM by then smile )
*
Is it possible to develop something similar in Celestia?
How much work is it?
Can it be borrowed from Space Engine?
***
http://en.spaceengine.org/forum/8-332-1
Transfer data to user can be made like transfer of textures are in Google Earth. The database is organized in octree, and this allows it to handle hundreds of billions stars and the interactive choosing of visible ones, like I've done in SpaceEngine.
The galaxy is subdivided into 1000 pc blocks. Each is subdivided into 8 childs 500 pc each. Each of them is subdivided into 8 childs 250 pc each and so on. SpaceEngine has 10-levels of octree. Each octree block has a star array associated with it. The magic is in the range of absolute magnitude for each octree level. The level 0 (1000 pc blocks) handle stars brighter than -5m abs mag, the level 1 handle -5...0m stars, the level 2 handle 0...+2.5m and so on. The step in magnitude is chosen so that each block contains approximately the same number of stars.
Render (or query for downloading) is based on limiting visual magnitude. For example, limiting visual mag = +6m. The Engine passes the octree and chooses the blocks that have visual magnitude of the most brightest block's star is less than +6m. At limiting magnitude +6m and infinite culling radius (sky render mode), the engine needs ~6000 stars in the Sun neighboorhood, at +7m ~50000 stars. This is just several tens of blocks, these will be downloaded (or generated) very quickly. The octree with magnitude culling allows avoiding querying of distant dwarf stars, that are already invisible. Once the blocks are downloaded from server, they may be saved in the disk cache. (For procedural stars, SpaceEngine doesn't use a disk cache, because generating of blocks are much faster than downloading them from disk.) Of course this system makes no sense for HIPPARCOS catalog with only 120k star, but for Gaia it will be neccessary (however, Gaia will be released in 2018-2019, our computers may have terrabytes of RAM by then smile )
*