cmodview: 3D mesh file utility for Celestia

[chris]

Cmodview is a graphical utility designed to help out Celestia add-on creators
who are working with 3D meshes. It can be used to inspect, optimize, and convert
meshes. Cmodview has basic material editing capabilities which can be used to change
textures and colors of mesh elements.

Prebuilt versions are available for Mac OS X and Windows:
Mac Disk Image: http://www.celestiaproject.net/~claurel/celest ... odview.dmg
Zip archive for Windows: http://www.celestiaproject.net/~claurel/celest ... -win32.zip

The code for cmodview is already committed to the Celestia SVN repository, so you can build it
yourself for any system capable of running Celestia. Currently, cmodview requires Qt 4.6 to build,
but this requirement will be relaxed so that cmodview can work with Qt 4.5.

Motivation

3D mesh files are an important component of many Celestia add-ons, but it's
difficult work getting meshes from modeling software into a form that's ideal
for Celestia.

Celestia supports two 3D file formats: 3D Studio (3ds) and
cmod. While most 3D modeling apps can export 3DS files, the format is a very
old one with many limitations: a 65535 triangle limit per submesh, 8.3
character texture file names, and no support for storing surface normals or
tangents. The cmod format is a better match for the capabilities of modern
graphics hardware, but the tools support for cmod has been fairly poor. Using
advanced features like emissive maps and normal maps required hand editing,

Features

The biggest benefit of cmodview is simply the ability to interactively
preview 3D models without having to launch Celestia. You can make changes to a
model inside of cmodview and instantly see the effects. Or, if you're
using some other program to modify a model, you can reload it in cmodview
with a couple keystrokes.

• Load cmod, 3DS, and Wavefront obj format files; save as binary cmod.
• Generate surface normals
• Generate tangents for normal maps
• Optimize models by eliminating duplicate vertices and aggregating meshes
• Apply normal maps, specular maps, and emissive maps to models
• View in OpenGL 2.0 mode or fixed function mode (fixed function is equivalent to
  Celestia's basic render path.) This is useful for checking out how a model will appear
  on both old and new graphics hardware.
• Show model statistics: number of triangles, vertices, materials, and submeshes
• Designed for Celestia: automatically looks for textures in textures/medres
  folder, loads dxt5nm compressed normal maps, etc.

Usage

• Select Open from the File menu to load a 3ds, cmod, or Wavefront OBJ file
• Left drag the mouse to rotate the model
• Right drag (or Alt+Left drag) the mouse to rotate the light sources
• Click on a mesh to select a primitive group. A primitive group is a
  set of triangles, lines, or points with a single material applied.
  The selected mesh is shown highlighted with a green wireframe.
• The material editor on the right shows the properties of the material
  assigned to the currently selected group. Changes to the material are
  reflected instantly in the model view window.

General Tips

• If the model doesn't appear to have any shading, it's probably because
  it doesn't contain any surface normals. Use 'Generate Normals' in the
  Operations menu to recover surface normals from the mesh geometry.
  
• Not all models have texture coordinates. If you add a texture in
  the material editor and nothing changes on the model, it's probably
  because of missing texture coordinates. This is something that has to
  be fixed in a 3D modeling program, as it is outside the scope of
  cmodview.
  
• Use 'Generate Tangents' (in the Operations menu) when you want to
  use a normal map. Texture coordinates are required to generate surface
  tangents. Don't generate tangents unless you're planning to add a
  normal map to a mesh, as they do increase the size of the model file.
  
• Generating normals produces lots of duplicate vertices. After
  generating normals, try using Uniquify Vertices (also in the
  Operations menu) to clear out the unneeded data.
  
• The Wavefront OBJ format has two big advantages over 3DS:
  it can store surface normals, and it doesn't suffer from the 64K
  triangle limit. (NOTE: material libraries for obj files aren't
  supported yet, so you need to add textures and colors yourself.)

Material Editor Tips

• For non-shiny surfaces like paper and chalk, disable specular lighting
  by setting the specular color to black.
• Create shiny, plastic like surfaces by setting the specular color to
  white and using a shininess of 60 or higher.
• Create metallic looking surface by setting the specular color to be
  the same as the diffuse color. For duller metal, use a low shininess
  around 5 - 15; for polished metal, choose a higher value.
• To make textured surfaces look metallic, set the specular map to be
  the same as the diffuse map and set the specular color to white.
• Emissive maps can be used to make some regions of a model appear self-illuminating.

Limitations

• Material libraries for Wavefront obj files are not supported yet.
• No way to export cmod ASCII files (just binary for now)
• Wildcard extensions (e.g. marsmap.*) for texture maps aren't supported yet.
• No way to to select lines and points yet

cmodview.jpg

(Centaur stage model from the LCROSS add-on by BrianJ)

[Cham]

Thanks a lot for this utility Chris.

Now, there will be a revolution on the asteroid front . Prepare for a new generation of high resolution models, like this one :

asteroid.jpg

asteroid2.jpg

[Reiko]

Thank yo much. This will be very much useful.

[Fenerit]

Wildcard extensions (e.g. marsmap.*) for texture maps aren't supported yet.

BTW
This doesn't affect the choosing of textures, anyhow; even for model already set with wildcard. The cmodviewer show just the model all black but it allow the change of texturing. (drop combo boxes)

[Fenerit]

Thanks a lot for this utility Chris.

Now, there will be a revolution on the asteroid front . Prepare for a new generation of high resolution models, like this one :

OBJ, eh?

[CAP-Team]

Wildcard extensions (e.g. marsmap.*) for texture maps aren't supported yet.

BTW
This doesn't affect the choosing of textures, anyhow; even for model already set with wildcard. The cmodviewer show just the model all black but it allow the change of texturing. (drop combo boxes)

That's also what I noticed, I can select the texture, but then the model goes black.
I never got the program to show the texture of any cmod.

[Fenerit]

Wildcard extensions (e.g. marsmap.*) for texture maps aren't supported yet.

BTW
This doesn't affect the choosing of textures, anyhow; even for model already set with wildcard. The cmodviewer show just the model all black but it allow the change of texturing. (drop combo boxes)

That's also what I noticed, I can select the texture, but then the model goes black.
I never got the program to show the texture of any cmod.

Just to "see" whether I've well-understood: after you do apply the texture upon the black model, do you have the model newly mapped? In my case the reply is yes. Otherwise, if you get the model still black, then could be matter of:

Not all models have texture coordinates. If you add a texture in
the material editor and nothing changes on the model, it's probably
because of missing texture coordinates. This is something that has to
be fixed in a 3D modeling program, as it is outside the scope of
cmodview.

[chris]

That's also what I noticed, I can select the texture, but then the model goes black.
I never got the program to show the texture of any cmod.

Just to "see" whether I've well-understood: after you do apply the texture upon the black model, do you have the model newly mapped? In my case the reply is yes. Otherwise, if you get the model still black, then could be matter of:

Not all models have texture coordinates. If you add a texture in
the material editor and nothing changes on the model, it's probably
because of missing texture coordinates.

I think Fenerit is correct here: you're trying to apply a texture to a mesh that doesn't have texture coordinates. Most of the models in the default Celestia package lack textures, as do a lot of the models in add-ons. Try changing the textures on a model that already has a texture applied.

There is a bug here, however... The surface shouldn't become black when a texture is added. In fact, cmodview should probably disable the texture map drop-down box when the selected mesh is missing texture coordinates.

--Chris

[duds26]

@Chris
Does this means cmodview uses the Cel(estia)Engine?

Does the CelestiaEngine needs to be adjusted to work in that different context or is it just strap controls+ui+load/save on it and done?

Why, oh why are you choosing cmod and not choosing/adding COLLADA as new format ?
Collada is very practical: supports the stuff you need (that 3ds format doesn't),
has content and a lot of programs/modellers/editors(Blender) can export to it.
Collada is an open standard royalty free content format.

It puzzles me to see such a choice, while clearly collada is a good solution and has some nice goodies:

https://collada.org/mediawiki/index.php/COLLADA_FAQ#What_are_the_major_features_of_this_format.3F

What are the major features of this format?

The COLLADA schema defines the XML elements and attributes that enable COLLADA to represent many features using these XML building blocks. The COLLADA 1.4.x feature set includes:

* Mesh geometry
* Transform hierarchy (rotation, translation, shear, scale, matrix)
* Effects
* Shaders (Cg, GLSL, GLES)
* Materials
* Textures
* Lights
* Cameras
* Skinning
* Animation
* Physics (rigid bodies, constraints, rag dolls, collision, volumes)
* Instantiation
* Techniques
* Multirepresentations
* Assets
* User data

COLLADA 1.5.x adds:

* Kinematics
* Boundary representations (B-reps)
* Geographic coverage
* Custom image initialization
* Math formulas
* GLES2 shaders

Collada's homepage on Khronos website: http://www.khronos.org/collada/

[selden]

Duds,

CMOD is designed for efficent loading of a model into OpenGL graphics cards. Collada is a 3D graphics interchange format based on XML. They have quite different purposes.

cmodview uses many of the routines that Celestia uses, but isn't identical.

[Fenerit]

Why, oh why are you choosing cmod and not choosing/adding COLLADA as new format ?
Collada is very practical: supports the stuff you need (that 3ds format doesn't),
has content and a lot of programs/modellers/editors(Blender) can export to it.
Collada is an open standard royalty free content format.

Duds26, I think Celestia's developer(s) does prefer to have a format on which to hold their own complete control, either for new features' addition or for being independent from features that could be irrilevant for Celestia or for which Celestia, on the whole purpose, is not designed for (e.a. skeletons for animations). Mind when the particle system will be added; one thing is to add the new directives upon a well-know proprietary format, another thing is to add it upon an exant format: probably this latter must be either assembled or disassembled in a way that developer(s) (of COLLADA in this case) could not agree and the work should be tedious and beyond the goals of Celestia's developer(s).
For what concern Blender, Blender already have a plug-in for exporting CMOD format.

[duds26]

I understand the reasoning behind it.
It's better to be independent by having an internally controlled format.

Really got me on the efficient loading thing, no way around it.

On the other hand, only a subset of collada that celestia and developers would find comfortable could be chosen.

@Fenerit, do you mean existant format instead of exant format?

Collada is an open and free format, only official endorsement, COLLADA is supported by..., costs money.

Now something that I find exaggerated, the mentioning about the particle system not being any good.
Particle system can be done with instancing and some other building blocks.
It's pretty nice described in the specs. I'm looking at page 199 and 211 especially. Page 211 and following are quoted below.
The specification has extra examples in appendix A.
The file is: https://www.khronos.org/files/collada_spec_1_5.pdf In the left and right alternating bottom corner of the pages, there is a little text:
April 2008
The specs about particles are out for quite a while now seems to me. Or do you mean something different?

instance_physics_model
Category: Physics Model
Introduction
Embeds a physics model inside another physics model or instantiates a physics model within a physics
scene.
Concepts
This element is used for two purposes: to hierarchically embed a physics model inside another physics
model during its definition, and to instantiate a physics model within a physics scene. It is possible to
override parameters of the contained rigid bodies and constraints in both usages.
When instantiating a physics model inside a physics scene, at a minimum, the rigid bodies that are included
in the physics model can be linked with transform nodes in the visual scene to let the physics animate
meshes that are being displayed. Similarly, if a rigid_body is kinematic instead of dynamic, an
application could take transform information from a targeted node that is being influenced by animation so
that the rigid_body is moved within its physics environment.
Additionally, it is possible to specify a parent attribute for the instantiated physics model. This parent will
dictate the initial position and orientation of the physics models (and correspondingly, of its rigid bodies).
The parent (or grandparent, etc.) can also be targeted by some animation controller, to combine key-frame
kinematics of nondynamic rigid bodies with physical simulation.
For details about instance elements in COLLADA, see “Instantiation and External Referencing” in
Chapter 3: Schema Concepts.
Attributes
The <instance_physics_model> element has the following attributes:
sid sid_type
A text string value containing the scoped identifier of this element. This value must
be unique within the scope of the parent element. This allows for targeting elements
of the <instance_physics_model> instance for animation. Optional. For
details, see “Address Syntax” in Chapter 3: Schema Concepts.
name xs:token The text string name of this element. Optional.
url xs:anyURI
Which <physics_model> to instantiate. Required. Can refer to a local instance or
external reference.
For a local instance, this is a relative URI fragment identifier that begins with the
“#” character. The fragment identifier is an XPointer shorthand pointer that
consists of the ID of the element to instantiate.
For an external reference, this is an absolute or relative URL.
parent xs:anyURI
Points to the id of a node in the visual scene. This allows a physics model to be
instantiated under a specific transform node, which will dictate the initial position
and orientation, and could be animated to influence kinematic rigid bodies.
Optional.
By default, the physics model is instantiated under the world, rather than a specific
transform node. This parameter is only meaningful when the parent element of the
current <physics_model> is a <physics_scene>.
6-14 COLLADA – Digital Asset Schema Release 1.5.0
April 2008
Related Elements
The <instance_physics_model> element relates to the following elements:
Parent elements physics_scene, physics_model
Child elements See the following subsection.
Other physics_model
Child Elements
Child elements must appear in the following order if present:
Name/example Description Default Occurrences
<instance_force_field> Instantiates a <force_field> element to
influence this physics model. See main
entry.
N/A 0 or more
<instance_rigid_body
target="#SomeNode">
Instantiates a <rigid_body> element and
allows for overriding some or all of its
properties.
The target attribute defines the <node>
element that has its transforms overwritten
by this rigid-body instance. See main entry.
N/A 0 or more
<instance_rigid_constraint> Instantiates a <rigid_constraint>
element to override some of its properties.
This element does not have a target
attribute because its
<rigid_constraint> children define
which <node> elements are targeted. See
main entry.
N/A 0 or more
<extra> See main entry in Core. N/A 0 or more

Here an example from page 216:

<physics_scene id="ColladaPhysicsScene">
<instance_physics_model sid="firstCatapultAndRockInstance"
url="#catapultAndRockModel" parent="#catapult1">
<!—Override attributes of a rigid_body within this physics_model -->
<!—and specify the initial velocity of the rigid_body -->
<instance_rigid_body body="./rock/rock" target="#rockNode">
<technique_common>
<velocity>0 -1 0</velocity> <!—optional overrides -->
<mass>10</mass> <!—heavier -->
</technique_common>
</instance_rigid_body>
<!—This instance only assigns the rigid_body to its node. It does no overriding
-->
<instance_rigid_body body="./catapult/base" target="#baseNode">
<technique_common/>
</instance_rigid_body>
</instance_physics_model>
</physics_scene>

About the copyright, it's allowed:

Khronos Group grants express permission to any current Promoter, Contributor, or Adopter member of Khronos to
copy and redistribute UNMODIFIED versions of this specification in any fashion, provided that NO CHARGE is made for
the specification and the latest available update of the specification for any version of the API is used whenever
possible.

[Fenerit]

@Fenerit, do you mean existant format instead of exant format?

yes, I mean existant.

Introduction
Embeds a physics model inside another physics model or instantiates a physics model within a physics
scene.
Concepts
This element is used for two purposes: to hierarchically embed a physics model inside another physics
model during its definition, and to instantiate a physics model within a physics scene. It is possible to
override parameters of the contained rigid bodies and constraints in both usages.
When instantiating a physics model inside a physics scene, at a minimum, the rigid bodies that are included
in the physics model can be linked with transform nodes in the visual scene to let the physics animate
meshes that are being displayed. Similarly, if a rigid_body is kinematic instead of dynamic, an
application could take transform information from a targeted node that is being influenced by animation so
that the rigid_body is moved within its physics environment.
Additionally, it is possible to specify a parent attribute for the instantiated physics model. This parent will
dictate the initial position and orientation of the physics models (and correspondingly, of its rigid bodies).
The parent (or grandparent, etc.) can also be targeted by some animation controller, to combine key-frame
kinematics of nondynamic rigid bodies with physical simulation.

The specs about particles are out for quite a while now seems to me. Or do you mean something different?

yes. Thats above are physical parameters concerning more that I've called "skeletons for animations" than particles system. Particles system is a way to add in effects free from physical parameters like inverse kinematics, nodes, bones, etc.
http://www.siggraph.org/education/materials/HyperGraph/animation/particle.htm

[Fenerit]

Another thing, since it was escaped to me:

About the copyright, it's allowed:

Khronos Group grants express permission to any current Promoter, Contributor, or Adopter member of Khronos to
copy and redistribute UNMODIFIED versions of this specification in any fashion, provided that NO CHARGE is made for
the specification and the latest available update of the specification for any version of the API is used whenever
possible.

that is what I supposed; there could be some modifications upon the specification in adding features that only Celestia requires.

[chris]

Regarding COLLADA...

The cmod format is already supported by Celestia, while Collada is not. Cmod files have been around for a few years now--this is not a new decision.

Collada has a lot of features that are not relevant for Celestia, such as physics, kinematics, and cameras. It would still be possible to have Celestia handle some subset of Collada, possibly just the geometry as Google Earth does. This would be useful, as there are few 3D modeling tools that export cmod files and more that export Collada. However, binary cmod files have the advantage of being more compact and much faster to load than the XML-based Collada files. It would make more sense to convert geometry only Collada files into cmod files ahead of time than to have Celestia directly load Collada. Collada is nice as an interchange format for authoring tools, but less than ideal as a final format for applications like Celestia. (This is what Selden was saying, I believe.)

Collada might be useful for its animation features. It would be useful to be able to define animation sequences within a 3D mesh file. It's not possible to express animation in a cmod file; rather than add animation features to the cmod format, it may be a better idea to use Collada, though more research is required first. The problem with Collada is that it's a 'kitchen sink' format: supporting all its features in Celestia would be a colossal effort, and partial support means that Collada files won't always work as expected. So some sort of Celestia-specific format that can be completely supported may still be a better idea.

Finally, Collada's particle systems are not a good fit for Celestia. Celestia allows the user to change the simulation time arbitrarily, while Collada particle systems won't work unless the time advances at some relatively steady rate.

--Chris

[ajtribick]

I do note in the FAQ linked upthread...

Are COLLADA documents included as part of games?

COLLADA is not designed to be used as a final game format. COLLADA allows 3D content to be created in any 3D package, exported to COLLADA format and edited with a variety of tools from different vendors. Once the content is finalized, it is usually processed into whatever format is most efficient for the game engine and hardware platform being used.

(emphasis mine)

Ignoring the use of the dreaded "g" word, looks like it isn't intended to be a file format used by the final application anyway...

[duds26]

Another thing, since it was escaped to me:

About the copyright, it's allowed:

Khronos Group grants express permission to any current Promoter, Contributor, or Adopter member of Khronos to
copy and redistribute UNMODIFIED versions of this specification in any fashion, provided that NO CHARGE is made for
the specification and the latest available update of the specification for any version of the API is used whenever
possible.

that is what I supposed; there could be some modifications upon the specification in adding features that only Celestia requires.

It's a good specification for getting inspiration from for an own format.
I know COLLADA isn't catered to Celestia's needs. That's why Celestia has cmod.
It is a useful format to get inspiration for own specifications/formats.

[chris]

Time to play the game, 'What's different about this image?' The screenshot is from the SVN version of cmodview.

mer-shadow.jpg

--Chris

[Fenerit]

Cmodview casts shadows...

[chris]

Cmodview casts shadows...

So that took about six minutes

But, yes, that's the difference. The shadows in cmodview will be introduced into Celestia once the code is perfected.

--Chris