Internationalization under Windows

[Joe]

translate_resources.pl does everything:
- translating the resource file (src/celestia/rc/celestia.rc -> src/celestia/rc/celestia_XX.rc)
- compiling them (src/celestia/rc/celestia_XX.rc -- rc.exe -> src/celestia/rc/celestia_XX.res -- link.exe -> locale/resXXX.dll)
- compiling the catalogs (po[2]/XX.po -- msgfmt.exe -> locale/XX/LC_MESSAGES/celestia[_constellations].mo)

I have to ask a dumb question , but it will help me getting on with the translation of Chinese version. What I am wondering are (1) where is the resource_strings.cpp file and (2) what the file extract_resource_strings.pl for? Thanks for helping

[Christophe]

I have to ask a dumb question :oops: , but it will help me getting on with the translation of Chinese version. What I am wondering are (1) where is the resource_strings.cpp file and (2) what the file extract_resource_strings.pl for? Thanks for helping

These aren't dumb questions at all! The internationalization process is a bit complex and makes use of the makefiles produced by automake which aren't available under Windows. So it's a bit difficult to get a clear view of the process when running only Windows.

resource_strings.cpp is a dummy cpp file. It is located in src/celestia/res/ and generated by extract_resource_strings.pl. It contains the translatable strings from the resource file.

The catalog file celestia.pot is generated by running "make celestia.pot" in the po directory. resource_strings.cpp is a dependency of the celestia.pot target and it is generated by the rule defined in the file Rules-win-resource which is included in the makefile.

Once generated, resource_strings.cpp is then used as input to xgettext along with the other cpp source files defined in POTFILES.in.

So "make celestia.pot" does everything automatically:
* extract translatable strings from
-- the data files (data/data.cpp in Rules-data)
-- the resource file (src/celestia/res/resource_strings.cpp in Rules-win-resource)
-- the KDE gui definition files (src/celestia/kde/rc.cpp in Rules-kde)
* build celestia.pot from the source files defined in POTFILES.in

Then running "make update-po" updates the existing po files with the new strings added to celestia.pot.

[Joe]

These aren't dumb questions at all! The internationalization process is a bit complex and makes use of the makefiles produced by automake which aren't available under Windows. So it's a bit difficult to get a clear view of the process when running only Windows.

Phew...,that is what I call i18na heavy dose to me. Christophe, thanks for your reply and it makes much sense to me now. I'll contribute my Chinese translation shortly. I have started but very slooow...

[Ajaja]

I built latest CVS-version of Celestia under Win32(MSVC). Internationalization (*.mo and res_<lang>.dll) works fine. But, there is something wrong with user-added non-english-named objects in *.ssc files in Solar System Browser window

winssbrowser.cpp:

Code: Select all

        item = AddItemToTree(treeView,
                             const_cast<char*>(_(world->getName().c_str())),
                             level,
                             static_cast<void*>(world),
                             parent);


In this case world->getName() return non-ansi chars (in UTF-8 ) and they don't converted from UTF-8 to CurrentCP() (WideCharToMultiByte was removed from AddItemToTree function)

[Joe]

I built latest CVS-version of Celestia under Win32(MSVC). Internationalization (*.mo and res_<lang>.dll) works fine. But, there is something wrong with user-added non-english-named objects in *.ssc files in Solar System Browser window

Which non-english language you are using? It could be caused by the *.ssc file. You may need to edit and save the *.ssc file in the UTF-8 encoding mode.

[Ajaja]

Which non-english language you are using? It could be caused by the *.ssc file. You may need to edit and save the *.ssc file in the UTF-8 encoding mode.

Russian in UTF-8

My temp solution of problem is change body.cpp:

Code: Select all

void Body::setName(const string _name)
{
#ifdef WIN32
    const char* lpszItem = _name.c_str();
    int length = lstrlen(lpszItem);
    LPWSTR wout = (wchar_t*)malloc(sizeof(wchar_t*)*(length+1));
    LPSTR out = (char*)malloc(sizeof(char*)*(length+1));
    int wlength = MultiByteToWideChar(CP_UTF8, 0, lpszItem, -1, wout, length+1);
    WideCharToMultiByte(CP_ACP, 0, wout, -1, out, length+1, NULL, NULL);
    name = out;
   free(wout);
#else
   name = _name;
#endif
    i18nName = _(_name.c_str());
    if (name == i18nName) i18nName = "";
}


It converts names of objects from UTF-8 to ANSI on stage of parsing *.ssc files.

[Christophe]

I built latest CVS-version of Celestia under Win32(MSVC). Internationalization (*.mo and res_<lang>.dll) works fine. But, there is something wrong with user-added non-english-named objects in *.ssc files in Solar System Browser window :(

You're correct. From the gettext manual:

Note that the msgid argument to gettext is not subject to character set conversion. Also, when gettext does not find a translation for msgid, it returns msgid unchanged -- independently of the current output character set. It is therefore recommended that all msgids be US-ASCII strings.

Arg! bind_textdomain_codeset was such an easy solution, too good to be true... I think we'll just have to make Celestia a Unicode application, either that or drop the MFC for another toolkit.

[Ajaja]

I think we'll just have to make Celestia a Unicode application

Would be great

But for the moment seems yet one small fix needed in winmain.cpp, something like:

Code: Select all

class WinSplashProgressNotifier : public ProgressNotifier
{
public:
    WinSplashProgressNotifier(SplashWindow* _splash) : splash(_splash) {};
    virtual ~WinSplashProgressNotifier() {};
    virtual void update(const string& filename)
    {
        bind_textdomain_codeset("celestia", CurrentCP()); //!!!!!!!!!!!!!!
        splash->setMessage(string(_("Loading: ")) + filename);
        bind_textdomain_codeset("celestia", "UTF8");
    }
   
private:
    SplashWindow* splash;
};


[Christophe]

I think we'll just have to make Celestia a Unicode application

Would be great

Well, in fact, since Windows Unicode applications use a UCS-2 coding internaly, we would still need to convert the gettext UTF-8 strings using MultiByteToWideChar.

I think rewritting the interface with Qt4 would probably be less work than converting the existing MFC interface to Unicode. With the added benefit of having a common interface on all platforms. But that won't be for 1.5.0...

But for the moment seems yet one small fix needed in winmain.cpp, something like:

Thanks for the patch, I'll commit it tonight.

[Christophe]

I've updated the zip file with the latest version including Joe's Chinese translation.

Font textures for Japanese (provided by Sui Ota) and Chinese are available. Simply unzip in the fonts directory. No modification of the config file is necessary, if the default fonts are set they will be loaded automatically.

[Joe]

Christophe

As the Celestia source is kept updating almost on a daily basis, the *.po file will also be regularly maintained up to date. My naive question is: if a *.po file is updated due to new translation being added or refined, do I have to send you the updated *.po file via email or there is a better way of making the file available for you to include it in the database?

I could have read more in the document to find the answer, but I am not sure I was reading the right pages . So thanks for any help

[Christophe]

Well, you don't have to send me an updated file for each new string appearing in CVS.

The way this is usualy handled is that before a new release a "string freeze" is declared after which strings can't be changed or added to the code. Translators then do their work and transmit the po files which are commited just before the release.

We are not that organized yet... and the main problem for us is that you need a full compilation framework to test a translation.

Anyway you are free to send me your po file as often as you want, e-mail is fine. If you have access to a web server that can be a better option since I could then automate the download and commit.

[Joe]

Many thanks to you Chris for the useful info about the submission of *.po files

We are not that organized yet... and the main problem for us is that you need a full compilation framework to test a translation

That was exactly my difficulties in the past few days when I was making and testing the translation. I am using WindowXP and still have not got around to making a full compilation framework to generate those related res_xx.dll files.

[Joe]

Christophe

I tried to run translate_resources.pl. What I did was:

--Install Perl5.8.8 in my WindowXP
--In Windows command console, under the po directory where translate_resources.pl file is located, I entered
perl translate_resources.pl

This is the error I got:
Unkown encoding 'CP' at translate_resources.pl file line 73

Help please?

[Christophe]

Sorry about that, I forgot to commit translate_resources.pl

Do a cvs update and you should be okay.

[Joe]

Thanks, Christophe. Mission accomplished

[Christophe]

I built latest CVS-version of Celestia under Win32(MSVC). Internationalization (*.mo and res_<lang>.dll) works fine. But, there is something wrong with user-added non-english-named objects in *.ssc files in Solar System Browser window :(

This is fixed in CVS. I didn't make Celestia a Unicode application, I simply resorted to using MultiByteToWideChar/WideCharToMultiByte in the cases where a non ASCII msgid might be returned by gettext.

Of course if you use characters not supported by your language in ssc files, there will still be problems as already reported by Sui Ota.

It is also now possible to use translated versions of:
- start.cel
- demo.cel
- guide.cel
- controls.txt
- COPYING (GPL licence file).

The translated file must be named with '_LL' before the extension, for example: start_fr.cel, and must be in UTF-8.

[Ajaja]

Thanks!!!

[Sui Ota]

OK.
Some feedbacks:

1.
I can't select Sol with translated name (while I could with English name "Sun") by pressing after Enter key.
I checked this in Japanese and French("Soleil").

2.
To be friendlier to translation, to do this is better, in celestiacore.cpp(lines 2762-2764).
Like the line of "Radius:"...

Code: Select all

 if (star.getLuminosity() > 1.0e-10f)
        overlay << _("Luminosity: ") << SigDigitNum(star.getLuminosity(), 3) << _("x Sun") << "??n";
    overlay << _("Class: ");


instead of

Code: Select all

 if (star.getLuminosity() > 1.0e-10f)
        overlay << _("Luminosity: ") << SigDigitNum(star.getLuminosity(), 3) << "x " << _("Sun") << "??n";
    overlay << _("Class: ");


Because with current version, the translation of this term is not anyhow suitable in Japanese...
(Maybe some other languages...)

3.
Some overlaying messages in winmain.cpp are garbled.
"Copied URL" and "Loading URL" should keep UTF8...?

Apart from internationalization:
Chris' bug fix for Ctrl+A has been missing in winmain.cpp.
http://celestia.cvs.sourceforge.net/cel ... .109#l2086
(line 2086)

[Christophe]

1.
I can't select Sol with translated name (while I could with English name "Sun") by pressing after Enter key.
I checked this in Japanese and French("Soleil").

Yes, the problem here is that the Sun is a special case since it's the only star with a localized named. Hopefuly, it also has an international name "Sol" which is used in scripts so we can safely translate "Sun" without breaking the scripts.

So this is fixed.

2.
To be friendlier to translation, to do this is better, in celestiacore.cpp(lines 2762-2764).
Because with current version, the translation of this term is not anyhow suitable in Japanese...
(Maybe some other languages...)

I didn't realize this could be a problem in some languages, this is fixed.

3.
Some overlaying messages in winmain.cpp are garbled.
"Copied URL" and "Loading URL" should keep UTF8...?

Yes, I was a bit overzealous here the strings must stay in UTF-8. Fixed.

Apart from internationalization:
Chris' bug fix for Ctrl+A has been missing in winmain.cpp.
http://celestia.cvs.sourceforge.net/cel ... .109#l2086
(line 2086)

I don't know how this sliped through the CVS update, anyway it's now fixed.

As usual, thanks for the fine report Sui