Bad addon at the Motherlode

[Cham]

I may sound rude and attract some crittics to myself with what I'm about to say. But it is part of my work, as a scientist, to make some crittics on bad addons which are trying to do something accurate and realistic. This is VERY important for Celestia's future health as a scientific tool.

On the extrasolar/data plots directory of the Motherlode :

http://www.celestiamotherlode.net/catalog/dataplots.php

there's a recent addon called "Association Cyg OB 2" from Miguel (third addon in the list). I was thinking about emailing the author, but I think it's a better idea to tell publicly what is wrong here, as an example of what may hurt Celestia. Many users may believe it's an accurate or safe addon, while it isn't. Of course, if I make a mistake myself here, please tell me and I'll retract with an apology.

Firstly, the readme file coming with the addon is in spanish only (or catalan, or whatever), and there's none in English. In the interest of all, there should be an English readme as well. Visibly, the author didn't care about communication. To me, this is a wrong indication about the validity of the addon itself.

Secondly, there are many double star systems in this addon, which are supposed to be realistic (it's not supposed to be some SF addon). Well, if you visit many of those double star systems, they just doesn't make any sense at all. Most of the companion stars are way too close from each other, they can't be stable or even be visible from Earth. And they don't move around their barycenter properly. How am I supposed to interpret this star addon ?

So, IMHO, this addon shouldn't be on the Motherlode, in a realistic directory.

I'm sorry, but we should be more carefull before sending any realistic addon to the public, and the guys at the Motherlode should be more carefull when it comes to select the realistic addons. This is about science, isn't ?

Now, I apologise if I sounded rude and clumsy. English isn't my first language, but at least, I'm trying to express myself in English so everybody could understand me here.

[Harry]

Sorry, but we don't have time to actually check the addons for scientific accuracy. I appreciate comments like yours, in fact we have to rely on such reports to fix any mistakes. I had hoped that more visitors would actually comment on addons, but this isn't the case.

Harald

[Cham]

To tell the truth, I was really disgusted by that addon, which isn't even aesthetically pleasant to look at.

There should be a realistic addon police. Like in real science, an independant team should verify the validity of any new realistic addon, before making it available to the general public.

This is in the interest of Celestia's credibility.

[Bob Hegwood]

There should be a realistic addon police. Like in real science, an independant team should verify the validity of any new realistic addon, before making it available to the general public.

This is in the interest of Celestia's credibility.

Pardon Cham, but perhaps you could volunteer for this service? I'm
positive that you have the knowledge, ability and interest to do this. And, I'm
almost certain that Harry would love this kind of help.

Maybe you could take on this task? I would, but I have nowhere near
the knowledge necessary to make these fine distinctions.

At any rate, I think you've already provided a valuable service here with
these comments.

Thanks very much, Bob

[Cham]

I'm interested in realistic addons (SF addons too, but that's not related). I will not hesitate to criticise any realistic addon which has something wrong or weird. But I can't do this alone. Especially, we can trash an addon on the Motherlode only because some guy (me) doesn't like it. Only a team can do it.

So in this particular case, someone else (Selden, t00fri, Evil Ganymede, or someone else ?) must download the said addon and play a bit with it. If he/she finds it's a bad addon, we then should ask Harry to remove it from the Motherlode. Too bad for the creator. If it happens he learn that we erased his creation, he will make the addon better next time, until it can pass the examination.

This is about realistic addons, so it must be serious stuff, or else Celestia will lose VERY FAST its credibility and seriousness.

[Bob Hegwood]

Cham,

Just one other thought for you...

Harry has gone to an awful lot of trouble to add the capability for Motherlode
Users to be able to add comments and rate an add-on. Have you used this
capability? Would be of service to other users there if your comments were
published on the Motherlode too.

Thanks, Bob

[t00fri]

Hi all,

I think that Cham rose a somewhat serious point. Perhaps not in the most diplomatic way...

Let me try putting it the way I can see the issue:

In Celestia code development, Chris reviews practically all code that is submitted into CVS by Celestia's developers. This is a (personalized) refereeing scenario and clearly provides the needed quality assessment!

The motherlode is a great GUI for centrally uploading add-ons, but it lacks precisely this aspect of quality assessment entirely. People have argued that every potential downloader will recognize immediately whether an add-on is good or bad. I have always doubted this!

The lack of any quality assessment, is by the way, my main reason why I will not upload any of my textures to Motherlode.

My proposition would be that we contemplate forming a (possibly anonymous) referee board, just like in case of publication of scientific papers. Some experienced people will be elected as members and Harry or Joe would address one member of that board in turn for reviewing any newly submitted add-ons before they are made available.

In case of criticism by a referee, the latter can get into contact with the submitter, and request improvements...or in the worst case turn the add-on down.


Bye Fridger

[Cham]

Yes Bob, I just used that feature 10 minutes ago. But it may take some time before it's published there.

[Cham]

t00fri,

I agree entirely with you.

You're a specialist of binaries. This addon contains many binaries. Can you check it and tell what you think of the parameters in the stc file ? They just doesn't make any sense to me. Stars are moving around their barycenter in some impossible ways.

[t00fri]

t00fri,

I agree entirely with you.

You're a specialist of binaries. This addon contains many binaries. Can you check it and tell what you think of the parameters in the stc file ? They just doesn't make any sense to me. Stars are moving around their barycenter in some impossible ways.

I could do lots of things but my actual spare time is really getting short. So this whole issue has to be agreed first by us and then well organized and the work distributed in a sensible manner. Otherwise I could never participate.

We need quite a number of people who would be willing to help regularly.

Bye Fridger

[rthorvald]

someone else (Selden, t00fri, Evil Ganymede, or someone else ?) must download the said addon and play a bit with it. If he/she finds it's a bad addon, we then should ask Harry to remove it from the Motherlode. Too bad for the creator. If it happens he learn that we erased his creation, he will make the addon better next time, until it can pass the examination.

Um, not thrash it, surely... But move it to a "junk" section, perhaps. I.e. Quality control without censorship...

One solution to the problem of lacking the manpower to check all addons, would be to have a "Pending" category. Everything that does not *obviously* belong to a section can reside there until enough people have downloaded it and provided feedback.

Another thing... It may be time to add a new top-level category to the ML - for things that aren??t fictional, but not entirely real either... I suggest "Conceptual". For things like my Sun, which are very borderline "real" ... Or that Pluto/Ganymede hybrid.

-rthorvald

[Cham]

A "non tested" or "yet unaproved" section on the Motherlode could be usefull for any realistic addon recently uploaded, with some kind of warning sign.

I must insist on the problem we may get sooner or later :

There's not enough manpower on the Motherlode to test every addon. And we will see more and more addons from many people around the world (with spanish only readme, and what else, chinese ?). Most addons will be about fantasy, SF or border line addons, some realistic addons with unreliable data, and so on. IT WILL GROW FAST, guys ! To me, the real problem-nightmare to come-is about realistic data.

One day, we will awake with a huge load of crap impossible to classify. And people may talk about Celestia as the unreliable astronomy software, not to be trusted. This is the price to have an open source, freely available software. We must act NOW !

[alphap1us]

Hi Fridger Cham and Bob,
I think everyone at the motherlode would be very happy to have any kind of additional quality control, but I don't think anyone who is an admin right now can spare any more time to perfrom this duty. Further, I don't see any reason to make a committee out of this. Anyone who is interested can download an add-on and contact the author about improvements, or fix the problem themselves and re-submit the add-on, or request that the add-on be moved to a different category, etc. As I said, we are always very happy to hear this kind of thing.

I must correct Fridger just a little bit when he says that the motherlode "lacks precisely this aspect of quality assessment entirely." It is true that we accept and submit almost anything that is posted to us, but it has been several months since Harald implemented a system of comments and ratings for just this very purpose. It has met with limited success, since many people do not click on the details page for this add-on. This may be because there is currently not much information there, since I originally did not collect the kind of information we are now able to display, so it's kind of a vicious circle in that respect.

I am strongly in favor of moving the star ratings to the main category pages, so that they are more visible, and making each link go to the details page, instead of immediately downloading the file. (I hadn't really brought this up on our list yet, as Fridger says, busy busy busy...) This would cost the user another click per add-on but would force them to see more inof about what they are about to download. If anyone has thoughts about this suggestion, pelase post them here.

So, as I have said before, if you want to help us with quality control, please make liberal use of the ratings and comments features. Also feel free to contact the author of the add-on for corrections or the motherlode helpers if there are info or category changes we need to make.

Cheers,
Joe

[rthorvald]

One day, we will awake with a huge load of crap impossible to classify. And people may talk about Celestia as the unreliable astronomy software

Here??s a simple solution to that one:
1)
Establish a set of strict guidelines for what a Celestia Addon is. Then classify everything
that fall short of these guidelines as fan art. Fan art should include anything that is not real,
regardless of quality, of course.

2)
Then establish a review board (like Fridger suggests) that only need to examine any submitted
work aspiring to be an Addon. Cuts down the work considerably.

-rthorvald

[Cham]

Very good idea, rthorvald.

"Fan art" is a good key word. Do others agree with this idea ?

[t00fri]

Hi Fridger Cham and Bob,
I think everyone at the motherlode would be very happy to have any kind of additional quality control, but I don't think anyone who is an admin right now can spare any more time to perfrom this duty. Further, I don't see any reason to make a committee out of this. Anyone who is interested can download an add-on and contact the author about improvements, or fix the problem themselves and re-submit the add-on, or request that the add-on be moved to a different category, etc. As I said, we are always very happy to hear this kind of thing.

I must correct Fridger just a little bit when he says that the motherlode "lacks precisely this aspect of quality assessment entirely." It is true that we accept and submit almost anything that is posted to us, but it has been several months since Harald implemented a system of comments and ratings for just this very purpose. It has met with limited success, since many people do not click on the details page for this add-on. This may be because there is currently not much information there, since I originally did not collect the kind of information we are now able to display, so it's kind of a vicious circle in that respect.

I am strongly in favor of moving the star ratings to the main category pages, so that they are more visible, and making each link go to the details page, instead of immediately downloading the file. (I hadn't really brought this up on our list yet, as Fridger says, busy busy busy...) This would cost the user another click per add-on but would force them to see more inof about what they are about to download. If anyone has thoughts about this suggestion, pelase post them here.

So, as I have said before, if you want to help us with quality control, please make liberal use of the ratings and comments features. Also feel free to contact the author of the add-on for corrections or the motherlode helpers if there are info or category changes we need to make.

Cheers,
Joe

Joe,

actually, assuming that I can spot BAD add-ons pretty fast, I checked quite a few "judgement" boxes for those bad ones. NOTHING! So my conclusion was that system is not practical (yet?). That's why I did not mention it explicitly above, although I was well aware of your attempts...

I think the only way to go is a fairly "formal" refereeing board" that is elected somehow. Otherwise we shall not get the 'workforce' together that is actually needed for such a challenging task.

Bye Fridger

[alphap1us]

While we are throwing out ideas, what about an icon that we only assign to add-ons that are rigorously sicentific? It could be a microscope or something similar, and would mean "This add-on conforms to the Celestia definition of a completely accurate astronomical representaion."

Of course, we seem to be side-stepping the issue of how we will possibly agree on the definition...

Cheers,
Joe

[t00fri]

While we are throwing out ideas, what about an icon that we only assign to add-on that are rigorously sicentific? it coudl be a microscope or something similar, and would mean "this add-on conforms to the Celestia definition of a completely accurate astronomical representaion."

Of course, we seem to be side-stepping the issue of hwo we will possibly agree on the definition...

Cheers,
Joe

My experience is that very few downloaders take the time to look at anything but the texture they want to get . Reading seems to be a pain in the brain in these days...

Sorry for being pessimistic here.

Bye Fridger

[alphap1us]

actually, assuming that I can spot BAD add-ons pretty fast, I checked quite a few "judgement" boxes for those bad ones. NOTHING! So my conclusion was that system is not practical (yet?). That's why I did not mention it explicitly above, although I was well aware of your attempts...

Do you mean you still got the message saying "not enough votes for a rating?" There is some lower limit on the number of votes counted before, so if you were the first to vote, it will not be displayed until others do so.

I think the only way to go is a fairly "formal" refereeing board" that is elected somehow. Otherwise we shall not get the 'workforce' together that is actually needed for such a challenging task.

Do you think it would be worth it to weed out bad add-ons in only one category? I think any incremental improvement would help, so I will need more convincing before I agree that a formal committee is necessary before any progress is made.

Cheers,
Joe

[t00fri]

actually, assuming that I can spot BAD add-ons pretty fast, I checked quite a few "judgement" boxes for those bad ones. NOTHING! So my conclusion was that system is not practical (yet?). That's why I did not mention it explicitly above, although I was well aware of your attempts...

Do you mean you still got the message saying "not enough votes for a rating?" There is some lower limit on the number of votes counted before, so if you were the first to vote, it will not be displayed until others do so.

I think the only way to go is a fairly "formal" refereeing board" that is elected somehow. Otherwise we shall not get the 'workforce' together that is actually needed for such a challenging task.

Do you think it would be worth it to weed out bad add-ons in only one category? I think any incremental improvement would help, so I will need more convincing before I agree that a formal committee is necessary before any progress is made.

Cheers,
Joe

Joe,

whatever your voting thresholds are: I must confess I personally do NOT give too much weight to "democracy" in this issue, since most voters simply will not be able to really judge scientific accuracy etc. If experienced people like Selden (or formerly Grant) , for example have a look, we are operating in an entirely different class...

I suggest we should think of competent people in the various relevant sectors: e.g.

Scientific assessment: Selden, Evil Dr. G, Cham, Spiff, ChaosSyndrome, Matt McIrvin, wcomer, me...
Texture quality: Runar, Jestr, Maxim, Brendan, DonEdwards (if back), JohnVanVliet, me...
Ficticous: Runar, Jestr,...
Scripting: Harry, Bob,...
Educational Frank,...

etc...

Bye Fridger