Generating CMODs from texture brightness

[Harry]

Hi,

I've been playing around with generating CMODs which consist of a n*n Grid with each point being elevated proportional to the texture brightness at that point. I just saw that selden was working on something similar a few months ago, so certainly not a new idea

As an example I modified the M83-Addon by Dave Mc (which alredy uses a non-flat model), my modified version can be found here. Here is a screenshot with the original M83 to the left, and my M83x to the right (but using both Addons is obviously a much better way to experience this):


This is just meant as an example to show how it works. By editing the texture used to create the CMOD the result can be influenced (e.g. smoothing, changing levels etc.).

What do you think - is this useful? It adds some depth, but in a fairly primitive way. It also has problems with the other side shining through the more you are looking from a side. And this doesn't give useful results for all types of galaxies and nebulae - galaxies which we look on from the side won't work well.

Harald

[selden]

My version is in Fortran 77. I'm guessing you wrote yours in a slightly more modern language.

If you invert the depths, it works reasonably well for some kinds of Nebulas, too, where dark areas are opaque dust clouds in front of excited gasses.

Unfortunately, the problem you mention is a defect in Celestia: DSC Nebula models aren't depth sorted. The models look better if they're defined as objects in SSC files instead.

[Harry]

My version is in Fortran 77. I'm guessing you wrote yours in a slightly more modern language.

Only slightly - a mere 26 years. And it has significant whitespace too! (Python)

It's still beta, but you can use it online here:
http://celestia.h-schmidt.net/create_cmod.py
The processing time allowed for a script is limited, so models with 128x128 points are probably the best I can offer online. Some performance optimization should be possible, but this won't help much (maybe 256x256). But the models don't really look better with higher resolutions anyway.

If you invert the depths, it works reasonably well for some kinds of Nebulas, too, where dark areas are opaque dust clouds in front of excited gasses.

Sounds good, but what happens at the borders?

I can easily add this as an option, but maybe this is better left to a preprocessing step (which should be done anyway).

Unfortunately, the problem you mention is a defect in Celestia: DSC Nebula models aren't depth sorted. The models look better if they're defined as objects in SSC files instead.

Just tried this, but I can still see parts of the other side (if the surface normal is less than 90deg away from my sight of line). Not sure why that is, but I wouldn't be surprised if I am missing something important while creating the models. Or maybe it's only because it's transparent?

Unfortunately this workaround "breaks" other aspects of Celestia like the rendering options or labeling, so I'll hope for a fix in Celestia :/

Harald

[selden]

but what happens at the borders?

I'm not sure what problem you're referring to. I found it easiest to put the individual vertices at appropriate distances and let the edges ripple.

Or maybe it's only because it's transparent?

Yup.
Celestia v1.3.2 and the 1.4.0 prereleases don't handle depth + transparency correctly. It's a bug introduced when Chris rewrote the model code. He did several quick hacks to try to improve it, but it's not fixed yet.

[Harry]

Celestia v1.3.2 and the 1.4.0 prereleases don't handle depth + transparency correctly. It's a bug introduced when Chris rewrote the model code. He did several quick hacks to try to improve it, but it's not fixed yet.

If I understand this correctly it's ok that the second layer becomes visible, it's just that transparency isn't handled correctly (a surface behind another should be less visible than it is now)?

Harald

[maxim]

A good first step. Your version is far too bumpy - but that's for demonstration only I believe. Also the bright center doesn't seem to be the highest elevation.

Since I saw the first version of this kind, I gave some thought to it. Here is how far I came until now:

While it seems to be a good idea to map brightness to altitude at first glance, there are some problems. First, the whole model should follow a more general structure that is given by the form of the arms and the core, rather than some bright spots. Secondly there are the dark nebula lines along the arms - as they cover all the light from behind, the are closest to us and therefore the highest elevation points on each arms - this assuption also seems to follow reality.

So a reasonable creation process would consist of multiple steps IMHO:

- if the picture is angled, rectify it by warping/streching.

- create a general 3D model - using Totis new galaxy rendering algorithm - by modifing the parameters until it fits the picture.

- Strip the picture into three layers: real background, bright areas, nebula structures. It seems to be difficult to distinct nebulae from background, but as the nebulae don't cover backlight totally, they always have a slightly violett or brown appearance when looked from close up. Color filtering should be able to greatly increase contrast for this task.

- Find a medium grey level for the bright areas, and add slight positive and negative bumps to the general model, based on that level.

- Add strong negative bumps for the background areas.

- Add noticable positive bumps for the nebula areas.


Don't know if this can inspire you. And I shurely can't see at the moment how to make that whole process to act without handweaving.


maxim

[Harry]

A good first step. Your version is far too bumpy - but that's for demonstration only I believe. Also the bright center doesn't seem to be the highest elevation.

The maximum height can be set when creating the CMOD. You can also make the image smoother using any image editor, or manipulate the height in specific areas. The heightmap used is included in the file M83x.zip (download link above).

Try it yourself: http://celestia.h-schmidt.net/create_cmod.py

So a reasonable creation process would consist of multiple steps IMHO:
- if the picture is angled, rectify it by warping/streching.

This is very important... and unfortunately some images won't be usable at all.

- create a general 3D model - using Totis new galaxy rendering algorithm - by modifing the parameters until it fits the picture.

I may be wrong, but I don't think it can be used for this purpose. Going from a simplistic 2D+height model to a real 3D model is a huge step - mainly because we simply don't have 3D information. Galaxies are the only exception, where the 3D structure can often be determined by categorizing the galaxy.
But even if you have a good 3D galaxy model it's not trivial to map the texure onto it in a way which looks good from all sides.

- Strip the picture into three layers: real background, bright areas, nebula structures. It seems to be difficult to distinct nebulae from background, but as the nebulae don't cover backlight totally, they always have a slightly violett or brown appearance when looked from close up. Color filtering should be able to greatly increase contrast for this task.

- Find a medium grey level for the bright areas, and add slight positive and negative bumps to the general model, based on that level.

- Add strong negative bumps for the background areas.

- Add noticable positive bumps for the nebula areas.

This certainly sounds like a good idea, though I am not sure what the result of these steps would be like in practice. However this is probably should be done in a preprocessing stage for the "heightmap" - automatically creating the different layers is probably very hard given the differences in source images.

Harald

[maxim]

I may be wrong, but I don't think it can be used for this purpose. Going from a simplistic 2D+height model to a real 3D model is a huge step - mainly because we simply don't have 3D information. Galaxies are the only exception, where the 3D structure can often be determined by categorizing the galaxy.

Why not? Can't we generate a 3D model from a greyscale/height model? We have to assume that the arms and the core are height, while the gaps are low - and that may miss the real 3d structure totally in some cases - but that's the only information we have. So if we twist the parameters of the algorithm, until the model exactly fits into the picture, we have a good starting point - or do I miss something?

But even if you have a good 3D galaxy model it's not trivial to map the texure onto it in a way which looks good from all sides.

Yes, the texture is warped along the bumps. But that's a problem that can only be handled by keeping the bumps quite moderate, I'm afraid.

This certainly sounds like a good idea, though I am not sure what the result of these steps would be like in practice.

Well, I don't know either. The described procedure was only a result of some brainstorming. One could create an example pic perhaps. I think I could create three greyscale/heightmap layers from an example galaxy, if you think you can alter your algorithm in a way to process these subsequently.

maxim

[Harry]

Why not? Can't we generate a 3D model from a greyscale/height model? We have to assume that the arms and the core are height, while the gaps are low - and that may miss the real 3d structure totally in some cases - but that's the only information we have. So if we twist the parameters of the algorithm, until the model exactly fits into the picture, we have a good starting point - or do I miss something?

I think we should separate these two things to make the discussion clearer:
- We have a 3D rendering of galaxies in Celestia, which (hopefully) will soon be improved to allow to render the various galaxy types differently (but they are still "grey blobs"). If you have the image of a galaxy you can find a matching galaxy model (like finding the type of a galaxy), and you probably could even modify the rendering paramters such that the rendering (number of arms, spacing of arms, etc.) matches the galaxy image closely. I assume this is what you had in mind. However I doubt that the model used for galaxy rendering is suitable to map a texture on it - for a realistic 3D model we would need texture information which also looks good from each side. So I don't think it's worth the effort to try and match the galaxy rendering to real images - simple parametrization, possible adding some hue, should provide nearly the same degree of accuracy.

- We can assume that the brightness of some object is related to its thickness, and based on that assumption generate a simple model which is pretty much 2D + height. But the resulting model isn't a real 3D model, it will always look imperfect when seen from a side. This isn't too bad for galaxies (which often are flat) or some nebulae, but doesn't give realistic results for many other objects. However this may still be preferable than having completely flat billboards (there is some danger in people assuming that the objects really look like the model). This method gives you the feeling of some depth, so it still looks good when seen from an angle, and this what my (or selden's) converer do.

But even if you have a good 3D galaxy model it's not trivial to map the texure onto it in a way which looks good from all sides.

Yes, the texture is warped along the bumps. But that's a problem that can only be handled by keeping the bumps quite moderate, I'm afraid.

But AFAICS Toti's new rendering algorithm doesn't produce bumps, it produces lot's of small clouds (someone correct me if I am wrong). It's certainly not made to map a texture on it. That's why I want to separate these two areas - the default galaxy rendering (grey blobs) and using replacement models (CMOD).

The described procedure was only a result of some brainstorming. One could create an example pic perhaps. I think I could create three greyscale/heightmap layers from an example galaxy, if you think you can alter your algorithm in a way to process these subsequently.

I am still not sure what you expect the resulting model to look like.

Currently it's just two layers (one facing in each direction) which do have some elevation above the default x-y plane, with the elevation derived from the brightness at the corresponding point in the supplied image.

Harald

[maxim]

I think we should separate these two things to make the discussion clearer:
- We have a 3D rendering of galaxies in Celestia, which (hopefully) will soon be improved to allow to render the various galaxy types differently (but they are still "grey blobs"). If you have the image of a galaxy you can find a matching galaxy model (like finding the type of a galaxy), and you probably could even modify the rendering paramters such that the rendering (number of arms, spacing of arms, etc.) matches the galaxy image closely. I assume this is what you had in mind. However I doubt that the model used for galaxy rendering is suitable to map a texture on it - for a realistic 3D model we would need texture information which also looks good from each side. So I don't think it's worth the effort to try and match the galaxy rendering to real images - simple parametrization, possible adding some hue, should provide nearly the same degree of accuracy.

- We can assume that the brightness of some object is related to its thickness, and based on that assumption generate a simple model which is pretty much 2D + height. But the resulting model isn't a real 3D model, it will always look imperfect when seen from a side. This isn't too bad for galaxies (which often are flat) or some nebulae, but doesn't give realistic results for many other objects. However this may still be preferable than having completely flat billboards (there is some danger in people assuming that the objects really look like the model). This method gives you the feeling of some depth, so it still looks good when seen from an angle, and this what my (or selden's) converer do.

Yes, we agree on that.

But AFAICS Toti's new rendering algorithm doesn't produce bumps, it produces lot's of small clouds (someone correct me if I am wrong). It's certainly not made to map a texture on it. That's why I want to separate these two areas - the default galaxy rendering (grey blobs) and using replacement models (CMOD).

But it produces a greyscale image. Altering the algorithm so that overlapping blobs generate more and more whiter areas will result in a model height map for this galaxy type (when seen from above) - that one smoothed is then the basis model when applying your brightness-to-height algorithm on it. Adjacent steps follow...

I am still not sure what you expect the resulting model to look like.

Hm, probably like a very flat Octopus lying on the beach, with the nebula areas arising from the arms like ripples.

Currently it's just two layers (one facing in each direction) which do have some elevation above the default x-y plane, with the elevation derived from the brightness at the corresponding point in the supplied image.

Yes, but my idea was to process the image throught the described steps, before running it throught the brightness-to-elevation code:

You split the image into three layers - make some processing onto every layer - remerge the resulting greymaps (plus the derived basic model greymap: makes four layers in total ) - and finally apply your algorithm to get the CMOD. The you map the original texture onto the CMOD.

maxim

[Harry]

Yes, but my idea was to process the image throught the described steps, before running it throught the brightness-to-elevation code:

OK, I finally understand what you had in mind.

Yes, that's probably possible - but I'd expect it to be less work to simply start with the image, and then painting it over to eliminate unwanted areas (background) and emphasize features...

Harald

[t00fri]

I am still not sure what you expect the resulting model to look like.

Hm, probably like a very flat Octopus lying on the beach, with the nebula areas arising from the arms like ripples.

YES, that's it a wet Octopus... (Are we talking about galaxies? )



Bye Fridger

[maxim]

I just tried to be descriptive.

maxim