NGA-GNS Location Data (+perl script)

Post requests, images, descriptions and reports about work in progress here.
Topic author
Zarius
Posts: 9
Joined: 02.01.2005
With us: 19 years 9 months
Location: Brisbane, Australia

NGA-GNS Location Data (+perl script)

Post #1by Zarius » 27.01.2005, 10:48

Hi all,

NGA: GNS GEOnet Names Server

NOTE: before anyone gets too exicited - there seems to be no population data for most of the locations I've seen so far, which means there's not much chance of weighting the importance and size variables. However there is latitude/longitude data.


Found this large source of location data a couple of weeks ago and have been playing with it for a while. I've seen it mentioned here once a while ago, however it was commented that the data was hard to get at. The site now has some download options however, check out the "HTTP Download" & "FTP Download" links down the side of the page.

This data has no licence restrictions, as noted on the main page and one can download a compressed text file for any country (excepting the US & Antartica, that data is elsewhere) or alternatively a 200mb zip file containing a massive 700mb text file of all the locations.

The files are TAB-Delimited making the easy to work with, but before you get too excited - note that there seems to be no population data for most of the locations I've seen so far, which means there's not much chance of weighting the importance and size variables. However there is of course the lat/long data.

The location data includes cities, but also extends to mountains, islands, rivers, undersea locations, railway stations, and a lot more. It is also specifies 'adminisitrative' regions (which translates to 'States' at least for Australia) for the locations.

I have created a perl script for my own use to parse this data into a format usuable by Celestia and incorporated a few features to get around the fact that Celestia really doesn't like loading 700mb worth of location data :twisted: I thought I'd share this script with others so they can make use of the data in whatever way they wish and it can be downloaded here.

The script requires you to know how to run a perl script (which requires you to have some form of perl on your system) but doesn't require any in depth knowledge of perl. I've gathered a heap of global variables at the top of the script that you can change to set the output directory & filenames, limit the countries output (if you are using the 700mb data file), and limit the features output (so you could for instance output only mountains in Tibet, Spain and France - or output only Railway Stations and Rivers in China & New Zealand, or whatever you fancy).

There is also a mode that outputs a couple of extra fields/variables ('SecondaryType' & 'Region') that don't currently do anything. I'm looking at attempting a little bit of coding in Celestia to try and allow it to enumerate the available types & regions (and probably 'countries' or something equivelent) into a dialog box that will let you specificy exactly what location types you wish to view). I'd also like to see locations shown as small dots which may show some text when you run the mouse over them, but that idea may be getting a bit too optimistic about my coding skills (plus it needs more thought about pros/cons of such a system).

Anyway, enjoy the script if you want to try it out - and although it currently just says 'Copyright' I am releasing it under GPL, just haven't updated the file yet :) I may make a few changes, namely I think there may be some size data I can play with to weight some of the features, can't check at the moment, any updates I make will be noted here. Oh, and suggestions and bug reports are quite welcome.

Some stats: approx. 66,000 locations for Australia - ~30 seconds to parse. Approx 5.5 million locations worldwide ~20 minutes to parse on my system.

PS: I was considering placing this in the development forum, as it kinda crosses over (being perl and whatnot), but the main topic is producing a set of location files, which I think qualify as an addon - let me know if I should move it elsewhere.

PPS: For those that don't know how to use perl, apoligies but I haven't found a free compiler for perl yet (which would make it easy for you to run) - in the meantime, search the web for perl and learn how to set it up on your machine or try ActivePerl.

Regards,
Zarius.

Topic author
Zarius
Posts: 9
Joined: 02.01.2005
With us: 19 years 9 months
Location: Brisbane, Australia

Post #2by Zarius » 30.01.2005, 16:14

Just a quite note to say that I've updated the script (if anyones used it) - it had a bug whereby it would only output to one file, that's now fixed and it correctly outputs to multiple files (by default - this can still be changed). The location is still the same, downloading from the link above should give you version 0.2 of the file.

No other major changes.
--
Zarius.


Return to “Add-on development”