New bugfix release 1.6.2-beta1

[onetwothree]

I will collaborate with Debian Astro team when 1.7 version is released.

[Chuft-Captain]

I held off for a long time on installing any of these 64bit builds because I was worried about the risk of breaking in some way my existing Celestia environment ... but 2 days ago I decided to bite the bullet and see what all the fuss was about, so I tried to install the 1.6.2 build from the beginning of this thread.

It seems that my fears were well-founded ... as I have had some issues with this installation (some minor , some serious )


First issue, the installer appears to be flawed:
(It appears to be missing 3 important steps in the process)

After accepting the license agreement, there should be an opportunity to provide an alternative install location ...

and an alternative shortcut location:

BUT the installer provides neither of these 2 screens, but instead goes directly to the "Additional Tasks" screen ...

Also ... after "additional tasks", there should then be a final "Ready to Install" screen, providing a summary of what will be installed, and where, and giving a final opportunity to either back out, or complete the install:

The installer presented none of these opportunities to customize the install. All I got was ...

• the license agreement,
• followed by "Additional Tasks"

at which point it simply went ahead and installed to a "mystery location" (a mystery, because of the missing "Ready to Install" screen)
I assume that it decided this location based on a registry entry (perhaps defaulting to the last location I installed to?).

In summary, the issue with the installer is simply the lack of transparency, customization, confirmation and control over what and where things are being installed, ... which may in fact have contributed to causing the more serious issues below....

2nd issue:
After tracking down where it had been installed to, I tried to run it, but it failed to startup (all I got was the splash screen, followed by what looked like a catch-all exception).


3rd (and most serious issue):
It appears to have broken my existing Win7 32bit installation environment in the following way ...

Some ScriptedOrbit and ScriptedRotation Lua code (which has worked perfectly for years) has suddenly stopped working.

I note that there have been some changes regarding LUA functionality ...

* Add support for Lua 5.2 and 5.3, drop Lua 5.0

so my immediate suspicion is that somehow installation (or a failed installation) of the new LUA code has somehow broken the OLD "circa Celestia 1.6.1" LUA code (Lua 5.1 IIRC) ... although this possibility seems a little outrageous ...

What may help me to work this problem is if someone (onetwothree?) can document the actions of the installer ...
eg. Does this version, or it's installer ...

• delete, or overwrite any existing DLL's in a pre-existing Celestia installation (32 bit)
• ... which might include changes which might affect the canonical order in which extras folders are parsed
• ...or changes WRT the Lua "require" statement
• write any changes / additions to the Windows Registry (related to either LUA or Celestia)

... and anything else you can think of.

My main concern is that this attempted install appears to have introduced side-effects in my previously perfectly functional 32bit version of Celestia 1.6.1 by breaking / invalidating existing LUA functionality.
(NOTE that I have already run the uninstaller in the installed folder (and then deleted the folder), but this has failed to rollback and resolve the issue with non-functional ScriptedOrbit's and ScriptedRotation's.)

Merry X'mas
CC

[SevenSpheres]

Chuft-Captain,

I can't duplicate your second or third issue. The basic installation of 1.6.2-beta1 works fine for me, and although Lua plugins don't work in 1.6.2-beta1, they still work for me in 1.6.1. AFAIK 1.6.2-beta2 will be released by the end of the year and will have Lua 5.1 support added back. The first issue definitely needs to be fixed though.

...and now I'll let the developers answer.

[onetwothree]

After accepting the license agreement, there should be an opportunity to provide an alternative install location ...

It present but only if you don't have Celestia installed already. That's a bug (or feature?) of Inno Setup tool. If you want us to use another tool we are waiting for your pull request.

followed by what looked like a catch-all exception

It would be great if you provide it. But I believe it's caused by absence of Universal C Runtime.

Added after 14 minutes 49 seconds:
A new pre-release of 1.6.2-beta2
A pre-release of the next beta version is available. Grab windows installer from here.

Changes since the beta1:

• compiled with SPICE support
• compiled with Lua 5.1 (LuaJIT used as with 1.7)
• incorrect markers in multiview mode are fixed (issue #468)
• favorites file is now looked in a proper directory
• splash screen is not top most now
• french and belarusian translations updated, incorrect translation of Seul in many languages fixed
• messages to console are converted to console (OEM) codepage (I suppose this is actual for cyrillic languages only)
• bundled libintl fixed to convert messages from UTF-8 to windows codepage in ui
• now it can be compiled as portable application (define PORTABLE_BUILD)

Known issues:

• On high DPI displays "Disable display scaling on high DPI setting" property must be set in a link.

Don't forget to install Universal C Runtime as it's required to run Celestia as it's built with Visual Studio 2015.

[SevenSpheres]

Hooray, the Lua Tools work again!

(I haven't forgotten about updating them, I'll do it eventually...)

[jujuapapa]

A new pre-release of 1.6.2-beta2
A pre-release of the next beta version is available. Grab windows installer from here.

Downloaded and installed this Morning.

Always the same anomaly : the datas of the frame "stellar browser" always begins by the Sun.
Even when you are near another star.
Why ?

On 1.6.1, the list begins by the closest star.

[onetwothree]

yes, something weird happens. we haven't touched this code.

Added after 5 minutes 29 seconds:
Actually we did. When 64 bit support was added for windows interface. Crappy windows!

[Markerz]

Here is macOS version of 1.6.2 beta2

macOS specific changes since 1.6.2 beta1 are:

1. Better support for Dark Mode on macOS Mojave+
2. Bookmark data from 1.6.1 is now preserved when migrating to 1.6.2
3. New authors are added to the about window
4. Remove a redundant fullscreen menu item (fullscreen support is provided by system)
5. Localization system is now in line with other platforms(gettext integration)
6. Lift the limit of number of alternative surfaces from 9 to 99
7. Revert to Lua 5.1

Celestia1.6.2Beta2.zip

[Chuft-Captain]

It present but only if you don't have Celestia installed already. That's a bug (or feature?) of Inno Setup tool. If you want us to use another tool we are waiting for your pull request.

Is "Inno Setup tool" a different tool to the setup tool historically used by Chris L, or is this just the 2015 version of the same tool? If so, it's seriously flawed.
I have never experienced an issue such as this with earlier Celestia releases. All previous versions of Celestia from at least 1.4 thru to 1.6.1 installled in a consistent fashion, and ALWAYS offered all choices in the setup process, confirmed user choices, and requested user confirmation before making any changes.
It's a very bad situation to have an installer blithely just plough ahead and install wherever the hell it wants without presenting customization options, and especially when there's no opportunity for the user to abort/rollback installation, so this does need to be resolved, but I'm afraid that I'm not currently in a position to assist in that respect.

It would be great if you provide it. But I believe it's caused by absence of Universal C Runtime.

Unfortunately I didn't take a screenshot at the time. All I can recall is that the error code was mostly 0's and had at least a single 3 digit in it.
ie. something like ... "03x000000"

Not very helpful, I know, but that's all I can recall.

The good news is that I believe I have resolved the issues with my broken LUA environment, and all seems well again in my existing Celestia installation.

CC

[onetwothree]

Is "Inno Setup tool" a different tool to the setup tool historically used by Chris L, or is this just the 2015 version of the same tool? If so, it's seriously flawed.

I used the latest available version from the Inno Setup tool site.

[selden]

Problem: Celestia v1.6.2b2 and Celestia v1.7 ignore ScriptedOrbit definitions

This Addon demonstrates a bug in Celestia v1.6.2b2 and Celestia v1.7, showing that ScriptedOrbit definitions are ignored.

Tha enclosed "scripted.html" provides a CEL:// URL which takes you to a viewpoint which should show a purple sphere (named "Scripted") dancing around a smaller yellow sphere (named "Test Object"). It works when using Celestia v1.6.1. Celestia v1.6.2b2 and Celestia v1.7 both draw the purple sphere at a fixed location, centered at the location of the yellow sphere, thus hiding it. If you type a Ctrl-W, Celestia will draw the objects as wire-frames, revealing the yellow object inside the purple one.

Two Celestia screengrabs are included in the Addon's zip file. The image "scripted_161.jpg" shows Celestia v1.6.1 drawing the two objects in separate locations. The image "scripted_162b2.jpg" shows Celestia v1.6.2b2 drawing the two objects in wireframe mode: both are drawn at the same location, with the purple object surrounding the yellow object.

The example ScriptedOrbit is taken from the Celestia Wikibook, expanded slightly to provide visible objects. See https://en.wikibooks.org/wiki/Celestia/Trajectories#ScriptedOrbit

Celestia v1.6.1 was installed using the SourceForge version at https://sourceforge.net/projects/celestia/

Celestia v1.6.2b2 was installed using the version at
[eta]https://bintray.com/celestia/celestia-builds/download_file?file_path=celestia-1.6.2-beta2pre1.exe

Celestia v1.7 resources were installed using download/file.php?id=6847&filename=CelestiaResources.zip
as provided by Markerz
[/eta]
Celestia v1.7 was installed using Janus' version at download/file.php?id=6839&filename=Celestia-5702-Vanilla.7z

scripted_orbit_bug.zip

demonstrates ScriptedOrbit bug

(60.8 KiB) Downloaded 407 times

Edited to add: this problem is seen running under Windows 10 1909 x64.
Additional edit: fixed link to 1.6.2b2 installer: [eta]...[/eta] = section edited.

[onetwothree]

Hmmm, very interesting, my first idea was that luajit is to blame, but the problem is deeper. I will take care of it.

[onetwothree]

Oh, the bug was unnoticed for almost 2 years here and for 5.5 years in fedora community. But it's fixed now.

[Janus]

@selden

Here is a present for you.
This is commit 5719 with the fix from 5726 put in.
I'll be updating the code I am experimenting with later.
Right now, I just wanted you to be able to test and verify this fixes everything for you.

Included is txf & ttf versions in x86 & x64 both.
Each has their own cfg file, and can be extracted into a working setup without interference.
Also included are fonts for the ttf version in the fonts directory, and your test object which is in the extras-standard directory.


Janus.

[selden]

Janus,

Thanks, a lot!

I've verified that your revised Celestia-txf-x64 (v 1.7, r5719) does indeed fix both the ScriptedOrbit and ScriptedRotation bugs which were reported by Chuft Captain above.

My test of ScriptedRotation did uncover a different bug, though: Earth is sometimes drawn red, indicating that there's a problem in one or more shaders. Since this shader bug is visible only in v1.7, I'll upload the test Addon to the v1.7 development thread, which is at viewtopic.php?f=4&t=18372

The shader bug is not seen in Celestia v1.6.2 beta2, although, of course, ScriptedRotation does not yet work.

Edited to add: I'm having some problems generating a test Addon and have some other commitments which probably will prevent me from uploading it tonight.

[Janus]

@Chuft-Captain

In reference to broken scripts, it is all about lua updates.
The single largest change, and what prompted me to turn back to lua5.1 from 5.3, is the replacement of getn with #.
Lua getn works consistently across tables and arrays both, where # only works on arrays.
With 5.2 dropping getn completely, any code that uses it is broken.
I looked at putting the getn back, but to many internal changes have been made.
The amount and type of work needed to do it exceeds my abilities with C/C++.

If you are using a celestia version with DLLs, replace the lua 5.3 dll with 5.1, and that should fix the script problems.


Janus.

[onetwothree]

Lua getn works consistently across tables and arrays both, where # only works on arrays.

Lua devs have good reasons to remove it. And even in 5.1 it's deprecated and users are encouraged not to use it in new code.

Added after 1 minute 23 seconds:

If you are using a celestia version with DLLs, replace the lua 5.3 dll with 5.1, and that should fix the script problems.

No, it will not. They are not compatible. Maybe one can replace 5.1 dll with 5.3 but not vice-verse.

Added after 1 minute 14 seconds:

I'm having some problems generating a test Addon and have some other commitments which probably will prevent me from uploading it tonight.

We can wait.

[Chuft-Captain]

@SevenSpheres: Thanks for your original efforts to replicate this issue.

I used the latest available version from the Inno Setup tool site.

My suggestion is to perhaps revert to an earlier *stable* version. It's just an installer after all, doing a very simple task, so unless there's a very strong justification for using the "latest and greatest" version, then the oldest version which "gets the the job done" is probably the best choice.

@Selden:
Thanks for putting together that handy little addon to demonstrate the issue.

Oh, the bug was unnoticed for almost 2 years here and for 5.5 years in fedora community. But it's fixed now.

Oh, indeed!
Have you considered incorporating some form of regression testing into your workflow? This almost certainly would have discovered the issue the moment it happened.
Chris L probably had little need for regression testing as he was intimately familiar with the code having written much of it from scratch, however despite that, I'm sure that he would have performed some level of regular regression testing.
With a new group of developers, it's even more vital to do so with such a complex piece of software, as no-one could reasonably be expected at this stage to have the same depth of knowledge as Chris L had of the codebase, and the potential side-effects of code-changes.

@Janus, and @onetwothree:
Regarding the lua5.1 vs 5.3 discussion... I'm not sure what the Lua dev's "good reasons" for removing getn() are as I don't follow the discussions at lua.org, however it seems to me to be a good and useful feature of Lua 5.1.
If Lua 5.3 breaks critical functionality in Celestia, or causes widespread impact on historical addons (eg. potentially Selden's Hale Telescope could be affected) and the need to rewrite existing lua and celX code in addons, then we should be careful not to rush into incorporating the latest version until all those issues are both identified, communicated to the community and/or resolved. (Regression testing would help to identify potential issues.)

IMHO, it's not always necessary for Celestia to incorporate the latest version of Lua immediately, simply because the Lua Team produce an update. Celestia development wrt. Lua integration needs to progress at it's own pace, with sufficient care and attention to any potential impacts on CelX and Lua based functionality within addons.
If Lua 5.3 is going to require major re-writes to lua-based addons, or break a lot of those addons permanently, then caution is required (not least, because ScriptedOrbits, and Rotations are a critical function within Celestia and have worked nicely with Lua 5.1 for many years).
So on a philosophical note ... my comment would be similar to what I said about the Inno Tool.
ie. "If it ain't broke ..." (or at least: tread carefully at a slightly slower pace.)

CC

Added after 17 minutes 49 seconds:

If you are using a celestia version with DLLs, replace the lua 5.3 dll with 5.1, and that should fix the script problems.

FWIW, when this issue first occurred, I did 3 things: 1. uninstalled the failed install, 2. rebooted, and 3. installed a clean version of 1.6.1 into a new location on a separate drive.
Somehow, that seemed to fix the problem, ... go figure!

[onetwothree]

My suggestion is to perhaps revert to an earlier *stable* version. It's just an installer after all, doing a very simple task, so unless there's a very strong justification for using the "latest and greatest" version, then the oldest version which "gets the the job done" is probably the best choice.

Who will test it?

Have you considered incorporating some form of regression testing into your workflow? This almost certainly would have discovered the issue the moment it happened.

Tests are code. Somebody should write them. Currently we are only 2 persons with very limited time.

Regarding the lua5.1 vs 5.3 discussion... I'm not sure what the Lua dev's "good reasons" for removing getn() are as I don't follow the discussions at lua.org, however it seems to me to be a good and useful feature of Lua 5.1.

It returns incorrect value in some cases.

Actually it's addon author's job to check Lua documentation and use only future-clean functions. I see no any good reason to use things that are clearly stated to be removed in future.

Added after 2 minutes 23 seconds:
But still, we stated several times that both 1.6.2 and 1.7 will use Lua 5.1 by default. While they have source-level compatibility with systems that don't have it.

[Chuft-Captain]

Who will test it?

In the first instance, whoever made the change would need to check that it works first ... followed by everyone else. This doesn't seem like a big issue, as the previous version will probably work fine.

Tests are code. Somebody should write them. Currently we are only 2 persons with very limited time.

I'm not trying to blame anyone for the bug not being noticed for years ... all I'm saying is that if we had a set of easily replicated baseline regression tests, in addition to the unit-testing you already (presumedly) do, it would help to catch this sort of unexpected side-effect in the future, whenever there are ANY changes to the code.
I have some ideas which may help you and the wider community in this respect, but I'm a little busy right now ... I'll come back to you about this later.

Actually it's addon author's job to check Lua documentation and use only future-clean functions. I see no any good reason to use things that are clearly stated to be removed in future.

I understand your position, however one of the great things about Celestia and one of the reasons for it's longevity and popularity is that Chris always respected the value of historical addons to the overall Celestia eco-system, and was generally quite careful not to make radical changes in the code if it meant breaking a lot of historical addons, in fact, sometimes he would sacrifice the "ideal" approach in order not to impose syntax changes in catalog files for example if that was going to break a lot of addons.
There is a history of established conventions and documentation on how to write ScriptedOrbit functions and the like for Celestia. If these latest LUA updates cause a break in those conventions, then perhaps this is one of those situations where a similarly slow and cautious approach is advisable ... at least until we have a better idea of how serious the impact on addons actually is.
Don't forget that an addon's original creator may no longer be around, and even if they are, it may be difficult or even impossible to re-write certain ScriptedOrbits to comply with the LUA 5.3 changes ...
At the moment it's a little unclear (at least to me) how serious and widespread the impacts will be, and whether it will even be feasible for all existing ScriptedOrbit functions to be made compatible and functional with the 5.3 lua version.

I know that I tend to adopt a couple of different styles in the way I implement ScriptedOrbits, but these are I think in line with the established conventions and whatever documentation that there is available. When I first encountered this problem, it appeared that some, but not all, of my functions were broken (perhaps all the functions written in one certain style were broken ... I'm not entirely sure).
I should also state that I've never *explicitly* called a getn() function in any of my scripts ... so unless there's something happening "under the hood" ... that specific issue does not necessarily account for the problems I encountered.