Allow Unicode and specifying font in printing text in .cel

[Rex]

So localizations is possible.
I want to translate some scripts into Chinese, but it doesn't work properly.

[dirkpitt]

I've been wondering about this too. I think since Celestia uses its own txf font files, a new txf with Chinese
glyphs in it would be needed. But I can't seem to figure out how to generate txf files in the correct
format. There was a similar question from a Japanese user before, but nobody answered the post.

I have some experience with Unicode and multibyte programming, so if Chris (or someone else?) could
give me some minimal documentation on whatever tool was used to generate the Unicode-friendly
txf files for Celestia 1.3.2+, I can help with the task of internationalizing Celestia.

[Harry]

Check out ttf2txf here:
http://www.shatters.net/~claurel/celestia/fonts/tt2txf/

Harald

[dirkpitt]

Hmm.. thanks. I saw ttf2txf before, but I wasn't sure if this was the program that Chris uses.
Hidden away, with no docs...

I'll see if I can get this to compile on OS X.

[Harry]

Just found this URL my mailinglist archive:
http://www.shatters.net/~claurel/stuff/txfutil.tar.gz

There is a README in there (and fonts, linux executables)

Harald

[dirkpitt]

Thanks, that helps.

After a few hours of hacking, I got Korean working:


(translation: "Welcome to Celestia!" aka "Hello World!")

The txf font I made only includes what sans*.txf includes, plus the Hangul syllable Unicode codepoints.
A 12-point txf with 11,514 glyphs required a 2048x1024 texture. That's not including any Chinese
or Japanese. If I were to add just the CJK Unified Ideographs (a subset of Chinese), it would require
an additional 20,991 glyphs. That would require a 4k txf at least. I'm not sure if this will cause problems
on video cards with low vram. Hopefully Celestia doesn't try to load the entire texture at once.

[Rex]

Thanks for what you have done.
But I'm not using Unix or Linux.
My OS is Windows XP.
And I do not have a gcc compiler.
I only have a Borland C++ compiler.
I have downloaded Freestyle and tried to compile ttf2txf.cpp.
But bcc says:
D:\Celestia\txfutil\txfutil>bcc32 ttf2txf.cpp
Borland C++ 5.5.1 for Win32 Copyright (c) 1993, 2000 Borland
ttf2txf.cpp:
Turbo Incremental Link 5.00 Copyright (c) 1997, 2000 Borland
Error: Unresolved external '_FT_Render_Glyph' referenced from D:\CELESTIA\TXFU
TIL\TXFUTIL\TTF2TXF.OBJ
Error: Unresolved external '_FT_Init_FreeType' referenced from D:\CELESTIA\TXF
UTIL\TXFUTIL\TTF2TXF.OBJ
Error: Unresolved external '_FT_New_Face' referenced from D:\CELESTIA\TXFUTIL\
TXFUTIL\TTF2TXF.OBJ
Error: Unresolved external '_FT_Set_Pixel_Sizes' referenced from D:\CELESTIA\T
XFUTIL\TXFUTIL\TTF2TXF.OBJ
Error: Unresolved external '_FT_Get_Char_Index' referenced from D:\CELESTIA\TX
FUTIL\TXFUTIL\TTF2TXF.OBJ
Error: Unresolved external '_FT_Load_Glyph' referenced from D:\CELESTIA\TXFUTI
L\TXFUTIL\TTF2TXF.OBJ
Error: Unresolved external '_FT_Done_Face' referenced from D:\CELESTIA\TXFUTIL
\TXFUTIL\TTF2TXF.OBJ
Error: Unresolved external '_FT_Done_FreeType' referenced from D:\CELESTIA\TXF
UTIL\TXFUTIL\TTF2TXF.OBJ

Please give me a binary version for win32.
Thanks.

[dirkpitt]

Thanks for what you have done.
But I'm not using Unix or Linux.
My OS is Windows XP.
And I do not have a gcc compiler.
I only have a Borland C++ compiler.
I have downloaded Freestyle and tried to compile ttf2txf.cpp.

By Freestyle I guess you mean FreeType. You'll have to link against the library, following the instructions
at the FreeType for Windows web site. Unfortunately it does seem rather complicated, but seeing as I'm
not a Windows dev I'm not sure how to help with compiling.

Once you do manage to get ttf2txf compiled, I can help. You'll need a text file containing the Unicode code points
you want in the txf font as hex codes; a good idea would be to append to the cel-subset.txt example file. I have in fact
already done this:

* File with 4E00~9FFF (CJK Unified Ideographs), 2500~257F (Box drawing), 2600~26FF (Misc symbols)
* Optional: Unix shell script for generating a range of code point hex values

To use chinese.txt, do something like ttf2txf -w 2048 -h 2048 -f chinese.txt -s 12 -o chinese12.txf chinese.ttf.
On my system, the resulting font has a very tight leading (line spacing), so you might want to experiment with the -g
parameter.



If you want more characters in your font, you might want to also include 3400~4DBF (Unified Ideograph Extensions A),
and possibly more. Depending on the number of code points, you may need a 2048x2048 (2k) texture,
or even a 4096x2048 (4k) texture. As I mentioned in my earlier post, this may be a problem because 4k fonts
may not even load on some video cards. For example, my 64MB Mobility Radeon 9700 is having problems with
4096x2048 txfs. What a bummer.

Perhaps Chris could elaborate on whether it's true or not that fonts are loaded as one big texture...

[Rex]

I have complied txfutil under WinXP with VC++ 7.
Now I can't get my simsun.txf working correctly.

One of the problems is that I cannot get line spacing:

even when I use "-g 20".

And I haven't seen a chinese word printed on Celestia when the showtxf displays many.

[dirkpitt]

Have you also localized all the text strings (not just the VC++ resource strings)?
To make this easier, Celestia has recently been updated to use GNU gettext,
meaning it supports .po translation files.

As a separate issue, you will also have to translate all your .ssc files too in order to
get Chinese planet, moon, and spacecraft names. This sounds like a very big project.

[Rex]

You see, I have edited a script.
There is a string "Welcome to ...".
I changes it to "Wel {} come to ..."("{}" presents a few chinese words.)
It displays "Wel come to ..."

[dirkpitt]

You see, I have edited a script.
There is a string "Welcome to ...".
I changes it to "Wel {} come to ..."("{}" presents a few chinese words.)
It displays "Wel come to ..."

Please make sure that you have saved your file in the UTF-8 encoding.

[Rex]

It's impossible to do so.
When I change the script encoding from ANSI to UTF-8 or Unicode.

And open it in Celestia.
It says :
'{' expected at the start of the text.

So ....

[Sui Ota]

Hello,

I succeeded to make a txf file included Japanese fonts using ttf2txf.exe compiled by Rex.
There was about 7,000 commonly used characters in Japanese, and they were within 2048x2048 size at 20pt.

Here is Japanese message using .CEL script:

(Sentences in the image are from Heike Monogatari, Japanese famous classics.)

The tool seems to be able to treat neither height nor width of over 2048 pixels.
So to deal with 20,991 characters in chinese.txt, font size is limited by about 14pt....


And....

One of the problems is that I cannot get line spacing:
[image omitted]
even when I use "-g 20".

It is probably a problem of font; I think it is better to use another font like Arial Unicode, which is bundled in MS Office.

It's impossible to do so.
When I change the script encoding from ANSI to UTF-8 or Unicode.

And open it in Celestia.
It says :
'{' expected at the start of the text.

So ....

Both notepad and wordpad of Windows seem to save the files as UTF-16 when we select "Unicode" in the 'Save as' window.
So I think it is good to use another text editor which can exactly select UTF-8.

[Joe]

Being not a programmer by all means, but I have tried very hard for long time to display some Chinese characters in cel screen, no success so far. When I saw those posts in this thread, I was excited and believed it could be done. I got the txfutil (ttf2txf) compiled and run on my PC XP and tried what Dirkpitt just explained, but I am still of no luck. Hope you programming specialists like Dirkpitt and Rex would keep this thread updated and resolve the problem. I would like to do my best to help, but need to improve my programming skills.

[Joe]

Well, no one came back to this thread recently . This is what I tried with my Windows XP system:

--Successfully compiled txfutil and got ttf2txf.exe worked
--Copied a simkai.ttf file from the Windows system fonts folder
--Copied chinese.txt file made by dirkpitt
--Put these three files in the same folder
--Used ttf2txf -w 2048 -h 2048 -f chinese.txt -s 12 -o chinese12.txf simkai.ttf and generated chinese12.txf
--Put chinese12.txf in the celestia's fonts folder
--Made chinese12.txf as the TitleFont in celestia.cfg file like TitleFont "chinese12.txf "
--Changed the "Welcome to Celestia" to Chinese charater in file start.cel
--Tried to save start.cel in UTF-8 format, but Celestia rejected, similar to what Rex had before.

Anyway when I ran Celestia after all the above work, I got the correct simkai font for all English charaters, but no Chinese charater displayed.

I am expecting helpful suggestions from your guys.

[Sui Ota]

Joe,

I got simkai.ttf, converted it to txf with ttf2txf.exe and chinese.txt., put it in \fonts\, and changed celestia.cfg properly.
This is result:

(I converted it with -s 20, because simkai.ttf I got has only about 7000 gryphs.)

At first, I save .cel file as UTF-8, and I ran the script, but the message "'{' expected at the start of the text. " was shown. The problem occured though it is saved as UTF-8!!!
And..., after a few minute, when I happen to save the file without BOM(Byte Order Mark), the script worked correctly.

So I think that the problem lies in saving the script file.
Other parts don't seem to have any problem.

- Sui

[Joe]

Thanks Sui, big applauses to you indeed. It works now well on my system.

[yaohua2000]

So localizations is possible.
I want to translate some scripts into Chinese, but it doesn't work properly.

It is what I want, it is the time drop .txf and use OS-native font rendering engine.

[Joe]

yaohua2000

It may be possible with Windows using OpenGL's wglUseFontOutlines function, but not sure it work with other OS.