Asteroids scripted extraction from MPCORB.DAT

Post requests, images, descriptions and reports about work in progress here.
Topic author
abramson
Posts: 408
Joined: 22.07.2003
With us: 20 years 9 months
Location: Bariloche, Argentina

Asteroids scripted extraction from MPCORB.DAT

Post #1by abramson » 17.01.2008, 11:32

Hi, all. I have written a PERL script that reads MPCORB.DAT (the offcial
table of asteroids orbital elements published by the Minor Planet Center
of the IAU) and converts its data to .ssc format, to be used in Celestia.
The asteroids can be filtered by several criteria. It's interesting. I
uploaded it to my page: mpcorb2ssc.zip.
I will appreciate comments and suggestions if anybody tries it. It is
thoroughly documented and commented.

It would be even better to use a celx and paint the asteroids in different
colors or with different symbols. But I do not speak celx. Even, forgive
my PERL, it has a strong FORTRAN accent! (Edit: new versions include
this.)

MPCORB.DAT contains nowadays nearly 350 thousand asteroids orbital
elements (be careful with your extraction, you may end up with a catalog
that clogs Celestia). You must download it independently, from the MPC
or one of its mirrors. Its a 70 MB file.

Cheers,

Guillermo

Current versions can be found here: (download with right click,
save as, if your browser loads the zip files in its window...)

Perl script: mpcorb2ssc.zip

Win32 executable: mpcorb2ssc-bin.zip

Sreenshots from Celestia: Trojans, Hildas, NEAs
Last edited by abramson on 19.10.2009, 19:09, edited 5 times in total.

danielj
Posts: 1477
Joined: 15.08.2003
With us: 20 years 8 months

Post #2by danielj » 17.01.2008, 13:50

It??s interesting.However,the file is TOO BIG and Celestia locks down.I tried the 90 MB file of MainAsteroidsBelt.ssc in Celestia 1.4.1 and it takes TOO LONG to Celestia to load,meaning that it??s too heavy.I don??t know which configuration would be necessary to load such a huge data...
But it appears that 2 GB DDR400 is not enough!

Topic author
abramson
Posts: 408
Joined: 22.07.2003
With us: 20 years 9 months
Location: Bariloche, Argentina

Post #3by abramson » 17.01.2008, 14:02

Danielj: In my post there is a warning precisely about the size of
ASTROB.DAT! I contains three hundred fifty thousand asteroids!

The purpose of a script like mine is to extract *some* of them, according
to a filter: name, size, high inclination, Centaurs, etc. Of course,
anybody can do as they like...

I insist: the extraction of *all* the asteroids in ASTORB.DAT
is useless in today computers and the current Celestia.


Guillermo

danielj
Posts: 1477
Joined: 15.08.2003
With us: 20 years 8 months

Post #4by danielj » 17.01.2008, 14:05

Ok,now I understand.Sorry...
I believe this kind of script is very useful,but how you know how many asteroids can be loaded in Celestia without crash?

Fightspit
Posts: 510
Joined: 15.05.2005
With us: 18 years 11 months

Post #5by Fightspit » 17.01.2008, 14:44

Daniel,

As you said before, loading too many asteroid make Celestia very slow to start but it doesn't mean that Celestia will crash. You can try several times with the script to increasing the number of asteroids you can load in Celestia.
Motherboard: Intel D975XBX2
Processor: Intel Core2 E6700 @ 3Ghz
Ram: Corsair 2 x 1GB DDR2 PC6400
Video Card: Nvidia GeForce 8800 GTX 768MB GDDR3 384 bits PCI-Express 16x
HDD: Western Digital Raptor 150GB 10000 rpm
OS: Windows Vista Business 32 bits

Topic author
abramson
Posts: 408
Joined: 22.07.2003
With us: 20 years 9 months
Location: Bariloche, Argentina

Post #6by abramson » 17.01.2008, 15:03

That would depend on your system speed and memory. I have just tried
with mine, extracting "the first N", so they are mostly main belt. I the
viewed them form a vantage point 20 AU from Sun, and above ecliptic. I
sped up time 10 million times, with asteroid labels on.

1000 asteroids: Fast, can see most of the labels clearly. Switching on
orbits, though shows a tight knot.

2000 asteroids: Fast, labels more difficult do discern.

5000 asteroids: Fast, most labels almost invisible, lost in a nice
doughnut. Here's where a celx would be nice, to put symbols on the
asteroids, instead of the labels.

10000 asteroids: Takes appreciably longer to load. Runs reasonably fast,
though. Trojans start to show up, making beautifull "shells" out of the
main belt. Another celx, to paint Trojans with a different symbol, would
be nice.

20000 asteroids: Loads even slowly. Runs als slowlier, but not unusable.
A solid mass of labels.

50000 asteroids: This is a massive 20 MB ssc file. Celestia takes 1 minute
and 20 sec to load. From the vantage point, doesn't run slowlier than
the previous. Celestia is taking up nearly 80 MB of my system's memory...

And I stop here, my scheduled backup has started, and the disk activity
would hamper Celestia even more.

My system here:
Core 2 Duo @ 2.33 GHz
Intel Board @ 1333 MHz bus speed
2 GB DDR-2 SDRAM @ dual channel 667 MHz,
NVidia 7300, 256 MB
Samsung SATA disk
Windows XP Pro
Celestia 1.5.0 built from CVS

Cheers,

Guillermo

Topic author
abramson
Posts: 408
Joined: 22.07.2003
With us: 20 years 9 months
Location: Bariloche, Argentina

Post #7by abramson » 17.01.2008, 16:23

Hi.

I have come up with a partially satisfactory solution to the selective
marking problem: the perl script now writes out a .celx script, with a
very simple format, marking all the filtered asteroids. This is almost all
my celx scripting knowledge, so I will welcome suggestions.

For each asteroid that satisfies the given criterion, the script does:

Code: Select all

print CELX "asteroid = celestia:find(\"Sol/$name\")\n";
print CELX "asteroid:mark(\"yellow\",\"disk\",5,0.7)\n";


Easy, isn't it? The you run the celx script from within celestia and voil? ! See the Trojans, (and they move so beautifully!):

Image

Give me some minutes so I update the readme, and I upload this new version.

Guillermo[/img]
Last edited by abramson on 01.03.2008, 16:08, edited 1 time in total.

Vincent
Developer
Posts: 1356
Joined: 07.01.2005
With us: 19 years 4 months
Location: Nancy, France

Post #8by Vincent » 17.01.2008, 16:45

Guillermo,

You could use the following celx script to mark all asteroids with size dependant markers:
http://celestiaproject.net/forum/viewtopic.php ... 5&start=40
@+
Vincent

Celestia Qt4 SVN / Celestia 1.6.1 + Lua Edu Tools v1.2
GeForce 8600 GT 1024MB / AMD Athlon 64 Dual Core / 4Go DDR2 / XP SP3

Topic author
abramson
Posts: 408
Joined: 22.07.2003
With us: 20 years 9 months
Location: Bariloche, Argentina

Post #9by abramson » 17.01.2008, 17:00

Thanks, Vincent. My knowledge of celx has just been boosted.

The new version, that writes out a celx which marks the selected asteroids, is ready.

I will implement some variations with different markings. Perhaps let the user choose a color for the celx, so different classes can be easily distinguished.

Guillermo

Topic author
abramson
Posts: 408
Joined: 22.07.2003
With us: 20 years 9 months
Location: Bariloche, Argentina

Post #10by abramson » 17.01.2008, 17:26

I am getting an error message when running these rather big celx's:

Code: Select all

Timeout: script hasn't returned control to celestia (forgot to call wait()?)

Do you know how to overcome this? I tried with a wait() at the beginning of the scriot but it does not do the job. Vincent, perhaps you know...

Guillermo

Avatar
selden
Developer
Posts: 10190
Joined: 04.09.2002
With us: 21 years 8 months
Location: NY, USA

Post #11by selden » 17.01.2008, 17:42

Celestia will abort any script which runs for more than about 5 seconds without letting Celestia update the screen. Celestia updates the screen during calls to wait().

So, call wait every 1,000 asteroids or so.
Selden

Topic author
abramson
Posts: 408
Joined: 22.07.2003
With us: 20 years 9 months
Location: Bariloche, Argentina

Post #12by abramson » 17.01.2008, 17:50

Thanks, Selden. I imagined that, and had already implemented precisely that workaround. If anybody has a better idea, let me know.

Behold the Near Earth Asteroids, with Amors in red, Atens in green, and Apollos in yellow:

Image

Guillermo
Last edited by abramson on 01.03.2008, 16:09, edited 1 time in total.

Avatar
Chuft-Captain
Posts: 1779
Joined: 18.12.2005
With us: 18 years 4 months

Post #13by Chuft-Captain » 17.01.2008, 18:33

I haven't tried this yet (I'm windows and don't have any sort of PERL interpreter on the system) but this looks like a pretty cool addon. Nice work Abramson.

(Some of those NEA's look as though they are 'not so near" :lol: )
"Is a planetary surface the right place for an expanding technological civilization?"
-- Gerard K. O'Neill (1969)

CATALOG SYNTAX HIGHLIGHTING TOOLS LAGRANGE POINTS

Topic author
abramson
Posts: 408
Joined: 22.07.2003
With us: 20 years 9 months
Location: Bariloche, Argentina

Post #14by abramson » 17.01.2008, 22:26

Thanks Chuft, sir.

Chuft-Captain wrote:(Some of those NEA's look as though they are 'not so near" :lol: )


Well, they weren't at the time of the snapshot. They cross the Earth orbit, though, so they are really NEAs. Not hazardous, thanks to resonant dynamics and such things. It is almost sure though that some potentially hazardous one, small and still undiscovered, belongs to one of those families...

Guillermo

Topic author
abramson
Posts: 408
Joined: 22.07.2003
With us: 20 years 9 months
Location: Bariloche, Argentina

Post #15by abramson » 18.01.2008, 17:24

Chuft-Captain wrote:I'm windows and don't have any sort of PERL interpreter on the system

Chuft and others:

For the benefit of Windows users who do not have Perl in their systems, I
have prepared also a "compiled" version of the script. It is the same
tools, packed with a very minimal version of Perl into an executable. It
will be here, always same version as the script:

http://fisica.cab.cnea.gov.ar/estadistica/abramson/celestia/mpcorb2ssc/mpcorb2ssc-bin.zip

Just unzip into a folder of its own, together with mpcorb.dat, and run
mpcorb2ssc.exe.

I have made small but useful changes since the first version, so update if
you please:

1. Writes out a celx script that, when run from Celestia, marks the
selected asteroids with custom colors.

2. The user can select the color each time, so different classes can be
painted differently in Celestia.

3. A brief explanation of the classes defined in MPCORB, so beginners
are not lost ("Cubewanos? What's THAT?")

Enjoy! (and let me know if something does not work, or can be improved).

Guillermo
Last edited by abramson on 19.10.2009, 19:11, edited 2 times in total.

Topic author
abramson
Posts: 408
Joined: 22.07.2003
With us: 20 years 9 months
Location: Bariloche, Argentina

new version 0.6

Post #16by abramson » 23.01.2008, 14:22

Hi all,

Minor but interesting new feature: selection by class can be combined
with selection by parameter (with a logical AND). So, for example, one
can choose all Trojans with inclinations greater than 20 degrees and
radius between 10 and 20 km. Nice.

Edit: You may also use partial names and Perl regular expressions when
selecting by name. For example lennon|mccartney|starr$|^harrison
will find The Beatles 8) .

Find the new version in the same place:

The Perl script: mpcorb2ssc.zip

The Win32 executable: mpcorb2ssc-bin.zip

And a new screenshot with a plot of all the Hildas, an incredible triangle 8O :
Image

Again, if anybody has suggestions or comments, they are most welcome!

Cheers,

Guillermo
Last edited by abramson on 19.10.2009, 19:12, edited 2 times in total.

Brendan
Posts: 296
Joined: 15.07.2003
With us: 20 years 9 months
Location: Bellows Falls, VT
Contact:

Post #17by Brendan » 24.01.2008, 00:34

I tried getting the zip files and they don't work, giving a 403 error when I try to get them. :cry: It would be a good idea to write something to merge in the daily updates, which I download everyday because the whole file is large.
Specs: 3500 AMD 64, 1GB RAM, PCI Express GeForce 7600 GS with 256MB, Debian Lenny on 250 GB drive, Windows XP Media Center on 250 GB drive

Avatar
Chuft-Captain
Posts: 1779
Joined: 18.12.2005
With us: 18 years 4 months

Post #18by Chuft-Captain » 24.01.2008, 00:50

Brendan wrote:I tried getting the zip files and they don't work, giving a 403 error when I try to get them. :cry: It would be a good idea to write something to merge in the daily updates, which I download everyday because the whole file is large.

I too experience this problem. The zip file is either corrupted, or inaccessible due to permissions on your server perhaps.

This looks very impressive. I'm looking forward to one day being able to observe the relationship between these triangular structures and the locations of lagrange points in my lagrange addon.
"Is a planetary surface the right place for an expanding technological civilization?"
-- Gerard K. O'Neill (1969)

CATALOG SYNTAX HIGHLIGHTING TOOLS LAGRANGE POINTS

BobHegwood
Posts: 1803
Joined: 12.10.2007
With us: 16 years 6 months

Post #19by BobHegwood » 24.01.2008, 02:59

Same for me Guillermo... BOTH links give the same error.

Access denied without proper permissions.

Just FYI.

Thanks, Brain-Dead
Brain-Dead Geezer Bob is now using...
Windows Vista Home Premium, 64-bit on a
Gateway Pentium Dual-Core CPU E5200, 2.5GHz
7 GB RAM, 500 GB hard disk, Nvidia GeForce 7100
Nvidia nForce 630i, 1680x1050 screen, Latest SVN

Topic author
abramson
Posts: 408
Joined: 22.07.2003
With us: 20 years 9 months
Location: Bariloche, Argentina

Post #20by abramson » 24.01.2008, 11:42

:oops: :oops: :oops: I'm very sorry, guys. Somehow the files lost their "read" permissions
yesterday. I have just corrected the problem.

The Perl script: mpcorb2ssc.zip (just 10.7kB).

The Win32 binaries: mpcorb2ssc-bin.zip (365.9kB, I removed example
screenshots from inside it to make it smaller).

The example screenshots are the ones I showed here: Trojans, Hildas, NEAs.

I am rather satisfied with the present functionality, and I am going on
summer vacations next week, so the current files will be stable for a
while. I will wait for your suggestions to make changes (unless some
of you discovers a terrible bug in the next few days :wink:)

Regards,

Guillermo
Last edited by abramson on 19.10.2009, 19:12, edited 2 times in total.


Return to “Add-on development”