How to deal with the mess !!

[Guest]

About five weeks ago I discovered a space simulation program which really hits me: Celestia.

Wow! Cool work Chris - and you others too, of course. So I started downloading all these textures and addons I found on the web, to enhance the basic distribution and enjoy my space explorations. Several days and about 1.5 GBytes later I had, what most of the newbies of you also found to have on their computers:

A LOT OF MESS !

Hundreds of textures, .ssc-files, models, trajectory-files and so on. No more overview of what I have and what not. Double and triple entries of the same space object, cluttering up the view in Celestia. Several versions of the same .ssc-files with different textures and models that had to be replaced, but are lost somewhere in the textures and models folders. In fact I was so occupied by keeping all these unter countrol, that I had not much time for enjoying any space travelling in celestia.

So I made a break and started to do a structural analysis, to find a way to organize the whole material in a manner, that could be suitable not only for me but for everyone of you out there. To my opinion this could also be a basis for a Addon CVS-Structure and, in extend, for an addon manager. Also there will be a compilation of my collected Celestia Tools and Addons available (about 1.5GB, fits easily on DVD) which uses this structure in the extras folder.

Finally here is my proposal of an organization structure for the extras-folder of celestia - with explanations:

The available celestia addons and textures groups together into different blocks. The most basic I've found is the level of reality, so the first level of subfolders are:

Code: Select all

     extras --+-- fantasy
              +-- fiction
              +-- real

========== FANTASY ==============================

'fantasy' is the folder for all these fantasy worlds you've created as well as all the spacecrafts from different SciFi movies and serials. By now I haven't found any other kind of fantasy-creations than solarsystems and spacecrafts, so that folder is divided this way:

Code: Select all

     fantasy --+-- solarsystems
               +-- spacecrafts

The subfolders structures then would be some of this (here my own compilation):

Code: Select all

     fantasy --+-- solarsystems --+-- Alrescha
               |                  +-- Altair
               |                  +-- Ringworld
               |                  +-- StarWars
               |                  +-- V380
               |                  +-- Vulcan Space
               |
               +-- spacecrafts --+-- 2001 Odyssee
                                 +-- Celestia1Spaceship
                                 +-- Space Lift
                                 +-- Star Trek Spaceships

Inside these named folders there would follow the usual structure of

'definiton-files',
'models'-subfolder,
'textures'-subfolder
...

The spacecraft-folder here is not subdivided, as it is inside the 'real'-folder (see later in this text), because often different crafts and objects, forming one common fantasy-universe, are grouped together in the definition files, an it's unlikely that some of them are individually or frequently altered or updated (This reminds me that I've never got 'Space Lift' to work. Celestia always freezes when it comes into sight).


========== FICTION ==============================

'fiction' is the folder for all space- or universe-structures that are sientifically proposed, foresaid or known but not individually identified or pictured yet. This includes for example 'black holes', 'pulsars', 'dark matter', 'energetic flows', 'protostars' and similar.

THIS FOLDER SHOULD HOLD VISUALISATIONS OF GENERAL CONCEPTS OF THE ABOVE PHENOMEN, NOT INDIVIDUAL OBJECTS !!!

This means that, for example, one or serveral different conceptual views of a black hole are kept here - NOT an individual black hole that may be once known by coordinates, mass and/or impulse - this would go into the 'real'-folder, even if the visualizing textures of it are fictious. The same refers for example to known extrasolar planets.

This folder holds the same subfolders as the 'real'-folder:

Code: Select all

     fiction --+-- clusters
               +-- galaxies
               +-- nebulae
               +-- singletons
               +-- solarsystems

These are the basic hierachical structures, that I've found are building space. They will be discused further down. At present there are only two folder that is data avaliable for, so this is as it looks like on my system:

Code: Select all

     fiction --+-- clusters --+-- M4
               |
               +-- solarsystems --+-- Black Hole
                                  +-- Dark Matter Simulation
                                  +-- protostar

As you see, it's not so easy to decide what goes where.

M4 is an existing cluster, so it should go (as stated above) into the 'real'-folder. But as all of it's stars are computer-generated and totally fictious in position, size and class, it'll stay here in the 'fiction'-folder.

The black hole and the dark matter are, in priciple, singletons - that means they should not be connected to any star. But celestia presently does not support the existence of objects other than stars on the same hierachical level. So the are in fact bound to a star and thus go into 'solarsystems'-folder until this changes.


========== REAL ================================

The folder 'real' holds the most of celestias addons. So it has to be subdivided according to the 'natural' hierachy of objects.

Code: Select all

     real --+-- clusters
            +-- galaxies
            +-- nebulae
            +-- singletons
            +-- solarsystems

In fact this is not quite correct. The universal hierarchy is about:

'galaxy clusters' -
'galaxies' -
'stars'
'nebulae'
'systems'
...

I think we can ignore 'galaxie clusters' as they are build from galaxies and have, beside of that, no (celestia-)usable center, rotation or other common structure. Well, you can of course break up all existing galaxy-definition files into the cluster parts, and then put these into different subfolders in a 'galaxie cluster'-folder. Perhaps this will happen in the future if the galaxy rendering is polished up. Also, for perfect hierarchy, all other nongalactic-subfolders seen here should go into a subfolder like this:

Code: Select all

     galaxies --+-- milkyway --+-- clusters
                |              +-- nebulae
                |              +-- singletons
                |              +-- solarsstems
                |
                ... others

I didn't do that, because it would be a permanently unused level of hierarchy. We do in fact (as far as I know) not know anything about nebulae, stars or any other detailed structures in other galaxies, so every non-galactic folder here refers automatically our galaxy. I had to break up several existing definition files - specially some of Selden's - to distrube them into this structural order. An example what's inside these folders:

Code: Select all

     real ---+-- galaxies --+-- M81
             |              .-- galaxies_extended.dsc
             |              .-- ...
             |
             +-- nebulae --+-- IC434
             |             +-- IC4703
             |             +-- M1
             ...           +-- M17
                           +-- M97
                           +-- NGC2237 Rosetta Nebula
                           +-- NGC2392
                           +-- ...

---------- SOLARSYSTEMS ----------------------------------------

Now we have to look more detailed onto the 'solarsystems'-folder. By now it contains only the sunsystem and a summarizing folder for all others:

Code: Select all

     solarsystem --+-- extrasolar
                   +-- sunsystem --+-- asteroids
                                   +-- comets
                                   +-- locations
                                   +-- moons
                                   +-- planets
                                   +-- spacecrafts

The sunsystem itself contains a folder for every object that could be in it. The same should be provided for every extrasolar system as soon as there is data for it avaliable. There is a hierachical break with 'moons' and 'locations' that will be discussed later. Asteriods are subdivided into some groups:

Code: Select all

     asteroids --+-- belts
                 +-- near earth objects
                 +-- singletons

'comets' aren't subdivided any further.

'locations' should hold some general location-definition files. Special location-files that probably are based on a certain alternate surface texture should go into a location-folder of the appropriate planet.

'moons' have the same hierachical level as planets, because I've discovered that for me it's much easier to remember a moons name than the planet he's revolving around. That's not consequent I know. Perhaps this will change somewhen in the future.

'planets' hold at least two subfolder:

Code: Select all

     planets --+-- surfaces
               +-- locations

'surfaces' for all alternativ surface textures along with the definition file.
'locations' as stated above.
Other things go directly into the 'planets'-mainfolder.


---------- SPACECRAFTS ----------------------------------------

So there are still 'spacecrafts' to look at. I think they could be completely divided into those groups (without gaps left):

Code: Select all

     spacecrafts --+-- ground stuff
                   +-- probes
                   +-- satellites
                   +-- space stations
                   +-- spaceships

For every type there are two subgroups. Example:

Code: Select all

     satellites --+-- fleets
                  +-- singles

So for example the shuttles are a fleet of spaceships, Globalstar is a fleet of satellites, and pioneer is a fleet of probes. If you like you can regard for example all geostationaries also as a fleet.

A part of my spacecraft folder looks like this:

Code: Select all

     spacecrafts --+-- ground stuff
                   +-- probes --+-- gallileo
                   |            +-- genesis
                   |            +-- huygens
                   |            +-- magellan
                   |            +-- mariner
                   |            +-- nozomi
                   |            +-- pioneer
                   |            +-- sirtf
                   |            +-- ...
                   |
                   +-- satellites
                   +-- space stations --+-- iss
                   |                    +-- mir
                   |                    +-- salyut
                   |                    +-- skylab
                   |
                   +-- spaceships

========== TEXTURES ============================================

Finally, also the textures in the texure folders should have an appropriate structure. Unfortunately celestia allows only for 'lowres', 'medres' and 'hires' subfolders. So if you want to keep control over your textures you have to store them all in an extra structure and copy these around that you want to use. I hope this will change in the future.

My proposed texture organizing structure would be like that:

Code: Select all

     textures --+-- lowres
                +-- 1k
                +-- 2k
                +-- 4k
                +-- 8k
                +-- 16k
                +-- 32k
                +-- 64k
                +-- ...

I hope it is obvious what the different subfolders mean. Inside the folders are the 'normal' textures of that size and two subfolders for the special versions:

Code: Select all

     4k --+-- DDS
          +-- VT
          .-- earth.jpg
          .--

This allows to differ easy between textures avaliable for NVIDIA-Systems and Non-NVIDIA-Systems. Of course there could be used more subfoldering to group several planets textures or anything else.


====================================================================

A last thing to mention:

The way for including and excluding addons for celestia should not be copying them around. Instead I rename the proper definition file(s) from *.ssc to *.ssc.no (and back) so it won't be recognized by celestia. Perhaps this could become a common way, as long as celetia doesn't provide a tree-structure where single addons can be checked on and off.


====================================================================

Well that was a long posting. There are some other plans, as to comment the
definition files in a javadoc manner with parser readable tags like

@ author
@ version
@ id
@ date
...

so it would be possible to autodocument addons, autodetect doubles and automatically compare two different celestia extra-folders for synchronizing them. But this will go in an extra posting.

So far, greetings & waiting for comments,

maxim.

[maxim]

Sorry - was my post above.
Wasn't logged in obviously.

maxim

[selden]

Maxim,

A quibble:

We do in fact (as far as I know) not know anything about nebulae, stars or any other detailed structures in other galaxies,

In fact, we know a lot about nebulae, stars and other detailed structures of some other galaxies. Do a web search on the LMC, for example!

Also, rather than reinvent the wheel, you should get involved with the Addon Manager discussion at news://celestia.teyssier.org/celestia.dev.addon_manager

[maxim]

Selden,

I have to apologize.
I located all these objects inside the milkyway.
So the hierarchy couldn't be shorted at that point.

But what I'm not trying to do is reinventing something. I proposed this folder structure not primary for an addon-manager (in fact such a structure is at least two steps before such a manager). It's thought mostly for handish organisation of addons. It could of course lead to a structure used in a repository. I tried to create it with care, so that there are as less gaps inbetween as possible, and that those gaps could be filled by just 'snapping' in, without reorganizing the structure.

One idea thereby was, that an addon-zip could contain the whole structure where it should be. And you just unzip every addon in the 'extra'-topfolder instead of copying it around guessing where it should go. To do this, you have to have a common basis.

Other ideas were the ability of comparing two installations for differences, or providing an addon folder-tree on a webside and let the user decide which parts he want to have as an on-the-fly-zipped download.

I didn't knew of any proposals anywhere in the forum, so I thought I have to do one - probably helping other newbies organize themselves.

greettings from oe,

maxim

[Tech Sgt. Chen]

Not that your proposed hierarchy is inadaquate Maxim, I just find it easier to use a modified version (personalized I suppose) of selden's propsed structure. Modified, or better still, personalized because I have an inordinate number of files under various program headings that already have a structured system committed to memory. Some of these files are accessed often, others not so... By doing it this way, I can still easily find files that would otherwise get lost in a sea of unlimited file structure types. I guess it's just a matter of preference.

[maxim]

Hello Tech Sgt.Chen,

I've missed that there is already proposed structure from selden. Where can I find it?

I would like to see an example of how you organize your addon data. In fact I think a common accepted basic structure would make things easier.

greets - maxim.

[Bob Hegwood]

Maxim,

Don't know if it will help, because I'm not exactly sure of what you're proposing...

However, the following is what I'm using to maintain my addons.

Within MY Celestia/Extras directory, I have simply added another directory called ADDONS, such that the resulting tree looks thus:

Celestia/Extras/Addons/

My ADDONS directory contains a separate sub-directory for EACH of my addons so that a directory called M42 can be found as follows:

Celestia/Extras/ADDONS/M42

My Models and Textures directories for the M42 addon are sub-directories of the sub-directory M42 so that the entire directory structure for M42 is contained on my machine thus:

Celestia/Extras/ADDONS/M42>SSC's, STC's and DSC's here
Celestia/Extras/ADDONS/M42/Models>3DS files here
Celestia/Extras/ADDONS/M42/Textures/Medres>PNG's and JPG's here.

I do this for ANY addon simply because it makes it VERY easy to work with the addon of my choice, and because it is EXTREMELY easy to move ALL of my addons from one version of Celestia to the next.

Maybe this helps?

Take care, Bob

[marc]

Maxim, Bob
If you look at the celestia.cfg file you can see that it is already possible to specify multiple extras directories for celestia.
Turning groups of addons on and off can be as simple as editing the cfg file.

[Bob Hegwood]

Marc,

Didn't know that. Thanks for the tip. I'll have a look at the config file.

Take care, Bob

[Tech Sgt. Chen]

Ditto!
Thanks marc!
I didn't know that either.

maxim,
My file structure is not all that different from what seldon proposed. I just tailored it to resemble a file structure that I've been using for all of my other files. My method is not necessarily better than the next person's, just easier for me to rummage through. The point that I was (not so eloquently) making was that each person will probably use a file structure that best suits their (cerebral) memory. seldon's structure example is located at:
http://www.lns.cornell.edu/~seb/celesti ... o.html#1.4

[maxim]

marc,

I do it just the other way round. Having three versions of Celestia installed, I use only one extras directory and modify the other .cfg files to point at:
'ExtrasDirectories [ "extras" "../Celestia/extras" ]' so I even don't need do copy around when installing another new version. Just modify the .cfg.
Unfortunately Celestia doesn't support absolute paths, so I can't put the extras direcory on my file server, like 'N:/Celestia/extras' and use it from other computers on the net (Should work with Linux but not with Win unter fat32).
I thought about what you said, but I'm switching my addons locally by renaming them to *.ssc.no, so I'm not too happy having two totally different locations for the same functionality.


bob,

what I was proposing above was an idea about a common agreed extras-directory structure. I my structure some examples would look like that:

Code: Select all

M42     -> extras\real\nebulae\M42\...(the usual stuff)
ISS     -> extras\real\solarsystems\sunsystem\spacecrafts\spacestations\iss\...
Voyager -> extras\real\solarsystems\sunsystem\spacecrafts\probes\voyager\...
Halley  -> extras\real\solarsystems\sunsystem\comets\default_comets.ssc
DS9     -> extras\fantasy\spacecrafts\Star Trek Spaceships\...
Altair  -> extras\fantasy\solarsystems\Altair\... (one created by Rassilion)

The last flyby asteroid -> extras\real\solarsystems\sunsystem\asteroids\near earth objects\2003_sq222.ssc

You see what I mean? Perhaps I should make a snapshot of my complete directory-tree structure in Explorer.


greets, maxim.

[maxim]

Tech Sgt. Chen,

Well, of course I'm absolutely not claiming to provide THE genius ideas. I just wanted to (perhaps) start a discussion which (also perhaps) could lead to some common agreement. Probably someone saying 'I never thought about that, but it's a good idea' or 'No, this is real bad idea - look how I manage it' maybe 'That doesn't interest me at all' and in the end all are stating that they could do it similar to others - like everyone is agreeing on how adressfields on letters are organized.

Maybe it doesn't lead to anything.

greets, maxim.

[Tech Sgt. Chen]

Don't misunderstand me maxim,
Your structure is much more organized than mine! Compared to mine, it is ingenious. Mine is just easier for me to deal with (maybe cause I'm simple minded in these respects) since I structue it in a way I'm familiar with(probably due to Conditioning). It's tantamount to having a cluttered desk and knowing exactly where everything is as opposed to, having someone clean it for you and being totally lost.

The discussion group is a great idea but, it already exists (which proves it's a good idea!) I wasn't trying to detract, just to explain (sideways sort-of) the Method To The Madness theory that (oddly) seems to work for some of us.

Post Script: You have three versions of Celestia installed??? You must either be a programmer or, "You are born again, hardcore!"

[maxim]

Tech Sgt. Chen,

if you ever feel you have to clean up your desk, feel free to remember this thread

Seems I'v completely missed the already existing discussion.

Post Script: You have three versions of Celestia installed??? You must either be a programmer or, "You are born again, hardcore!"

Well, I DO software development, and these installations where just fast consecutive updates as I detected them - and the old rule: 'Never uninstall anything as long as you don't now the update works as expected'.

greets from oe, maxim