Import API

[MibTaz]

Hmm. Well, The feature request is under users, but this seems more like dev, so I'm posting it here. If I'm wrong, please move the post to its appropriate location :p

I would like to make a feature request. A library API for dynamically linking libraries at runtime. I realize that this is not an easy task. The reason is because I've seen some libraries like NOVAS and SOFA (and others) would add features to Celestia that would make it more robust, but including them directly creates more problems than they would solve. What I'm proposing is an API like Apache, where you can load a dynamic library at run time. Benefits include extensibility, flexibility, and robustness without being tied to external source. Detriments include building a new API, shared object bugs corrupting the running executable, and a point for buffer overflow / shared object attacks.

I suppose the question that comes to mind is "Why?". I know I thought about that before posting this. The reason is that are many people out there don't know C++ but would like to add "xyz", and others that hate to maintain local changes but want to stay current with the application (me). I wouldn't mind writing a translator library, but I at shun maintaining changes that are useless to the majority of users. An import API would allow people to extend Celestia without maintaining a separate source tree, and allow the extensions to be used by anyone without requiring them to duplicate the work.

As I see the API, it would be configured in celestia.cfg, with something like:
SharedObjects = "true"/"false"
SharedObjectsDir="path"
SharedObjectsAllowed="name"[[,]"name"]/"all"/"none"

SharedObjects would designate if any were allowed.

SharedObjectsDir designates the directory for the .SOs or .DLLs.

SharedObjectsAllowed designates either a list of .sos/.dlls to load, or all in the directory, or none in the directory. This way 3rd party libraries wouldn't be confused with Celestia libraries.

The person that wanted to use a 3rd party library would then write a translator, and store it in the SharedObjects directory. The translator library would be responsible for setting up any required buffers or stack frames, registering the public routines with Celestia, and calling the intended 3rd party routine. This could be simply calling the function, or as complex as converting data types and post processing the results.

The API itself would parse the config, find the libraries (if any), load the library, and call the initialization function for it. The translator library would then verify and set up anything it needed from the initialization function, and register it's public functions. It would call Celestia with a prefix/pseudo class name to register the functions with. Celestia could then return a status designating if the pseudo class name is OK or already in use. If it is available, the translator library would then register each of its public pseudo members with Celestia. Celestia would check to see if the pseudo class has been set up, and if so attempt to add the function to the pseudo class. Celestia would return a status of the pseudo member, either OK or not OK (possible failures would be already registered, not able to add the name to the export list, etc.)

Once this was complete, the library would be available through pseudo_class:pseudo_member from the LUA interface. This would allow maximum extensibility while requiring minimum dependence on 3rd party modifications, and maximizing portability.

[GlobeMaker]

Hi MibTaz,

The open source software is ready for you to implement your ideas. Your description is a general interface description. I hope you do this work, with detailed discussions from knowledgeable developers (not me). I can give some comments as a lightweight programmer.

Your .dll interface needs hooks in the following classes and functions :

renderer class draws things. To draw new things takes understanding.

observer, celestiacore, simulation, universe,
getStarCatalog()->
star->setPosition(barycenterPosition);
CachingOrbit
sim->getActiveObserver()->getPickRay(pickX * aspectRatio, pickY);

return UniversalCoord(uc.x + (BigFix) v.x,
uc.y + (BigFix) v.y,
uc.z + (BigFix) v.z);

PlanetarySystem* system = body()->getSystem();

situation.translation = situation.translation + getVelocity() * dt;


After you understand what hooks are needed for these items, you can modify the source code.

_______________________________________________

I like your plan. Here is what I would do with it :

executables (.dll) would use the hooks to load and unload stars and planets and textures as needed.

.dlls would coordinate several computers and monitors for simultaneous execution for stereo depth perception.

Perl and C programmers could write executables that work in Celestia.

Internet connections would be allowed for remotely prepared data streams.

Output of display data for movie quality post processed displays. Instead of outputting to screen only, a data stream would be stored on disk for post processing.

[MibTaz]

I'm willing to make the changes, but only if they're something that would be rolled into the release version. One of my qualms is putting in 80+ hours and finding out that it's a local change. I was thinking about it as more of an extension tool for scripting routines, but if it was done right there's no reason that it wouldn't be able to handle more elaborate functionality like registering rendering routines, on the fly orbit recals and even on demand texture downloads, etc. Of course I'd want to make the changes against 1.4. If Selden or another one of the primary developers (forgive me if you fall into that category GlobeMaker, I'm new to the forum) thinks it's worth doing/ thinks it's worth adding, I'm willing to implement it.

BTW GlobeMaker, the ideas you propse make the interface more intriguing :)
I'm hoping to get some more positive feedback from the primary developers, this is starting to sound like fun :)

Cheers

[GlobeMaker]

Hi MibTaz,
You say you are new here, so let me explain some basic facts. I am an unimportant newcomer. Selden is not a developer he is the moderator. The main person is Chris Laurel, and rest assured that he will read this within two days. Selden will read your ideas within a half day, although he may not comment until I stop posting or your thread falls silent.

The key people are listed in the Menu : Help : About .
Chris Laurel
Clint Weisbrod
Fridger Schrempp
Christophe Teyssier
Grant Hutchison

Most of those people will read your words here within 16 hours. For your work to be accepted into the 1.4.2 or later official versions of Celestia requires their approval, especially Chris. He might comment here, or he might not.

Best wishes, good luck, your supporter and minor player, GM

[MibTaz]

LOL I should have read the help, caught by RTFM.

Thanks for the info

[MibTaz]

Hmmm. I suppose either I was not clear (a very definate possibility), life has been been busy for the developers (another very definate possibility), or this is something that isn't very usefull for/to people. In any case, I'll check back in a day or so.

In case I wasn't clear, the idea is to build an API that allows people to add functionality to Celestia via a seperate DLL or SO, without requiring a localized change. This way 3rd party programmers, like myself, could add something, say the SOFA fortran library or an on-the-fly texture download without needing to rewrite the Celestia source every time there was an update to it, and other people could use the change without needing to patch and recompile the source.

If Chris and the primary programmers want to do this, yay. If they like the idea and would rather someone else impliment it, I can do it, and to whetever specs they require. I'd just like to know so I have an idea, if this is worth pursuing. Doing the work doesn't bug me, but if it's a localized change that no one else will see or use, it's not worth the effort.

Thanks for listening

[chris]

In case I wasn't clear, the idea is to build an API that allows people to add functionality to Celestia via a seperate DLL or SO, without requiring a localized change. This way 3rd party programmers, like myself, could add something, say the SOFA fortran library or an on-the-fly texture download without needing to rewrite the Celestia source every time there was an update to it, and other people could use the change without needing to patch and recompile the source.

It's not difficult to load libraries on the fly. But, adding the appropriate hooks to Celestia to make these libraries actually useful (to support a feature such as on-the-fly texture download) requires some careful design. Or would you be satisfied with being able to access the libraries strictly through Lua scripting?

--Chris

[hank]

Lua 5.0 includes a function called "loadlib", which takes as its arguments the file name of a dynamic library and the name of a function in the library. It loads the library and returns the function, which can then be called from Lua code. This should be available on platforms which support the 'dlfcn' standard (e.g. Windows,Linux,MacOSX). I don't know if it currently works in Celestia or not. Presumably if not, that can be fixed.

If you just want to call external libraries from Lua user scripts, this should be sufficient. But if you want to extend Celestia's internal operations, you'll need to have the necessary hooks in the Celestia code, as Chris indicates.

A useful approach might be for Celestia to allow an optional Lua "event handler" to be registered for each object, along with an "event mask" indicating which event types are of interest. The Lua function would be called with a dictionary specifying the details of each applicable event. It would return a boolean indicating whether Celestia should also perform its normal processing for the event. This approach is similar to how Lua can be used currently to customize keyboard input.

- Hank

[MibTaz]

Chris,
Most of what I want to do is access library mathematical functions and data lookups. Access via LUA scripts is what I wanted I think Hank's idea of using loadlib would suffice.

Following Hank's suggestion, I tried using the example in the LUA 5.0 ref ( which is print(loadlib()) ) in Celestia 1.4.0-winx, I recieved the following message:
[string: "<path>/ll.celx:"];3: attempt to call global 'loadlib' (a nil value)
Just to make sure I wasn't fat fingering the sequence, I tried it in a lua 5 console, and it complained about args being wrong, which is what it was supposed to do.

I think accessing libraries through scripting is more reasonable. That'll allow alot of robustness without redesign.

Thanks for the feedback, from everyone
Tony