New 3D file format

[chris]

I'm working on a new 3D file format for Celestia, along with tools to convert from 3DS format. Why a new format when there already dozens? Here are my reasons:

Efficiency:
Most 3D file formats are not designed for fast rendering on 3D hardware. A significant amount of preprocessing is required in order make 3DS models closer to optimal for hardware rendering. Why not have a file format where this preprocessing has already been done, with triangles in strips ordered to make best use of the vertex cache?

Explicit normals:
The 3DS format doesn't allow normals in files; they have to be generated from vertex data, and this can lead to unsightly creases--take a look at most of the asteroid models. With sufficient processing of the mesh data, the crease can be removed. But why do this at run time? The 3DS format also lacks the capability to control the 'edge threshold' for smoothing. Look at the artifacts near the outer edge of the high-gain antenna on almost any spacecraft model.

More information stored per-vertex:
I want a file format that can store surface tangents for bump mapping. And multiple texture coordinates. And additional data that might be specific to a particular shader.

Animation:
Finally, I'd love to eventually add motion to models . . . Jack Higgins has several versions of Galileo--one just deployed from the space shuttle, another in cruise mode with the probe attached, and a third version without the probe. It would be great to be able to create a model with solar panels that could unfold at a particular time and a probe that detached (well, disappeared right at the time a separate probe object appeared.) With a custom model format, this would be possible.

I'm working on this concurrently with a lot of other Celestia development work. It will take a while before I have something working, but I believe strongly that this is a worthwhile effort. People are creating some great models, and I'm upset that they don't look as good as they potentially could in Celestia.

I'm posting this partly just to rant, but I would like to hear some feedback. Is there an existing 3D file format out there that would at least satisfy my first three requirements?

(Even when I have the new format ready, Celestia will continue to support 3DS models. I believe in keeping the barriers to Celestia content development as low as possible, so if you don't feel like using the conversion tools, you won't have to. One of the reasons that there are so many add-ons is that Celestia uses standard file formats that can be created directly by a wide range of programs--JPG, PNG, 3DS, and text files.)

--Chris

[selden]

Chris,

Your idea sounds very good to me.

I'm sorry I don't know enough to be able to suggest an existing format.

On my rather slow system, Celestia pauses for a noticable amount of time whenever it has to load a new model. I'm not sure how much of that is due to the model itself and how much is due to loading surface textures. I'm sure both contribute.

Hopefully it'll make it possible to implement depth sorting of facets within Nebula models!

[fsgregs]

I love the idea of animation, Chris. For one thing, it might give us the capability to put lights on spacecraft (steady or blinking). I could also envision the potential for a more interactive experience using Celestia. Imagine flying up to a fictional rotating space station and attempting to dock with it while its shuttle bay doors opened in front of you. NASA would flip out for one and a lot of other users would get a real kick out of it.

Naturally, Celestia would still provide the pleasure of pure science and Astronomy for those who don't want this kind of bell and whistle, but I feel the purpose of Celestia is to bring a user into space; otherwise, why have it? I would love to see movement and animation added to that experience (along with sound).

Sorry, but I have no knowledge of graphics processing and cannot help with the format for a new graphics approach. I did, however, find a site that claims to have information on "every graphics format on the planet". Its web address is

http://www.tutorialhunt.com/default2.asp?tree=2

Frank

[HankR]

Chris,

You might want to take a look at OpenSceneGraph (http://www.openscenegraph.org/). I'm not suggesting you rewrite Celestia's rendering code to use it, although you might want to evaluate that option. But it may be useful to steal some of OSG's 3D model stuff. (Not that we couldn't reinvent it here, of course).

Just a thought.

- Hank

[JackHiggins]

Chris

I have to agree with everything you said above. Models just doesn't show as well as they could in celestia, using only the 3DS format. Hopefully a new format would reduce the file sizes too...

Animation, if it is possible, would be a real bonus too. Galileo is a prime example. As the only spacecraft xyz included with the Celestia install package, it would be great for beginners if they could view the entire mission from start to finish. (Launch, deployment, booms extending, xyz segment, and final destruction in Jupiter's atmosphere)

2 other things:

1) Would it be possible to specify an axis for a model to rotate around? Most of the models in Celestia either don't rotate, or it is wildly innacurate.

2) If a fixed rotation axis is possible, (e.g. around a HGA) would it be possible for this axis to be pointed at the earth, and "track" the earth during the mission, much like the user can? This is normally what would happen in reality, so why not in Celestia?!
For the cassini mission, it would be great if you could instruct Cassini to point at, and "track" Titan during their encounters, as it'll be mapping the surface at these times.

Even if these aren't possible, a new format is definitely a good idea.

[HankR]

I would love to see animation in Celestia. It would be great to watch the Mars Exploration Rovers landing on Mars next January, for example. First the landing capsule separates from the cruise stage, then it enters the Martian atmosphere and makes a fiery descent. Eventually the parachutes open, the lander slides down its tether, the air bags inflate, the retro-rockets fire, the the tether is cut and the lander drops to the surface, bounces several times, and finally rolls to a stop. The airbags retract, the lander petals unfold, and the rover drives off onto the Martian surface to begin its exploration...

Well, perhaps that's a bit ambitious for an initial implementation. But it's fun to imagine!

And really, all that's required is to be able to arbitrarily specify the location/orientation and shape/appearance of each object at any given time point. That sounds simple enough!

And, oh yes, we'll need sound effects, of course...

- Hank

[JackHiggins]

Chris,

If there is to be animation etc in Celestia, the problem with objects appearing completely in front/behind other objects will have to be sorted out too... Otherwise things will look very strange when deploying from the shuttle or other stuff like that, when objects are in very close proximity.

[PhillipT]

Chris,

this is just a thought, would it be worth looking at Microsoft's Direct X format - it offers a chunk of what your looking for including animation. While obviously its designed for Direct X, you could use the file parsers etc from the Direct X OpenGL wrapper project to ensure cross platform support. available at :

http://sourceforge.net/project/showfile ... p_id=44365

Just a thought - since its a file format that targets application areas that are looking for the similar feature sets...

the wavefronts .obj offers some features in terms of storing normal information and it also offers surfaces as well. The open scene graph (openSG) project has a good .obj parser and loader

A good list of 3D geomertry formats and dev. information on them can be found at:

http://astronomy.swin.edu.au/~pbourke/geomformats/

-- PT

[edited for clarity...]

[chris]

Chris,

this is just a thought, would it be worth looking at Microsoft's Direct X format - it offers a chunk of what your looking for including animation. While obviously its designed for Direct X, you could use the file parsers etc from the Direct X OpenGL wrapper project to ensure cross platform support. available at :

http://sourceforge.net/project/showfile ... p_id=44365

Just a thought - since its a file format that targets application areas that are looking for the similar feature sets...

I'll take a look at this code . . . I did consider the DirectX format, but it looks like a horrible format to read. It's completely overdesigned, with a ridiculous template extension mechanism and a ton of legacy crud like texture wraps. I find it all very distasteful :>

the wavefronts .obj offers some features in terms of storing normal information and it also offers surfaces as well. The open scene graph (openSG) project has a good .obj parser and loader

The problem is that while .obj does allow normals to be specified in a file, it doesn't permit surface tangents or other 'extended' vertex attributes. It also stores vertices as lists of indices (point, texture coordinate, normal), which is flexible but requires preprocessing in order to convert it to an efficient organization for hardware.

A good list of 3D geomertry formats and dev. information on them can be found at:

http://astronomy.swin.edu.au/~pbourke/geomformats/

Yes this does seem to be *the* 3D file format site . . . Unfortunately, I still can't find a format that satisfies all of my requirements.

--Chris

[PhillipT]

Hi Chris,

not suprised you find it distasteful. The X format is essentially a data dump for the architecture - X files just hold data and echo closely the directX SDK. When you parse an X files using the file parser routines,the data is read, an appropriate D3DX object (meshes, skin information etc) or buffer (materials, effect instances) is constructed and handed back to you. When i first looked at the file format by itself i thought it was pretty yucky!!, but when i considered in conjunction with the sdk (or in this case the opengl wrappers) some of it makes more sense (or at least it did to me) though its not particularly elegant, it works well. The template extension mechanism is somewhat overengineered but can be useful for experimenting with support for new features. Though is a lot of stuff which shows the evolution of the file format and architecture...

I'd be very interested to see what you come up with and might be able to help with writing file converters and pluggins for some modelling apps if that would be useful...


--PT

[Guest]

hi there,
youre stuff looks cool. i just found the site. will look to dl the app asap.
maybe youd like to experiment on formats using some of the starbasec3.com spaceships models....we make a mojoworld version, could make a library for you too...

larryr@starbasec3.com