Neural network image upscaling

Discussion forum for Celestia developers; topics may only be started by members of the developers group, but anyone can post replies.
Topic author
john71
Posts: 1009
Joined: 10.08.2016
With us: 8 years 4 months

Post #21by john71 » 10.11.2021, 21:38

fyr02 wrote:This result is not much better than the 4K one.
all that is being done is edge sharpening and noise reduction on the clouds, which is both unrealistic and not worth the extra file space that comes with the 16K.

I'm sorry, but it seems you don't understand upscaling. 4x upscaling should produce a much worse quality when zooming in (try it in Gimp). Cupscale produces a better and 4x upscaled image.

I was able to create a good quality 64k texture from a 14k Jupiter texture.

https://celestia.space/forum/download/file.php?id=11595&filename=Jupiter+64k-1.png&mode=view

Added after 26 minutes:

Normal image enhancing is like this:
New vs old-detail-comparison.JPG


But Cupscale's AI recreates the image (zoom in to see the VAST difference):

New vs old-detail-comparison.JPG-4x_BSRGAN.jpg

Avatar
fyr02
Posts: 249
Joined: 19.04.2019
Age: 19
With us: 5 years 7 months
Location: San Diego, California (United States)

Post #22by fyr02 » 10.11.2021, 22:07

No, this is not what I am saying.
I'm saying that in the case of this texture, it is not justifiable to upscale the Neptune texture to 16K.
If you keep continually upscaling it to whatever resolution, eventually the data will be overfit and in the end you are not increasing the quality by much (in fact, it is the opposite, you are slightly decreasing the accuracy) when you upscale past a certain point.
In this case, it's not much worth it to upscale Neptune past 6K. Such miniscule improvements (which Cupscale does such as keeping edges, reducing noise, which is the point of the ai upscale) are not representative of the real texture, and as such upscaling the texture past 4-6K is useless.

In the case of Jupiter, when the original data was 14k, this limit of data improvement lies somewhere between 32K and 64K. Upscaling your textures (such as in this case, where 2K was upscaled to 16K) will not improve the quality past a certain point. all that is being done is increasing the weight of the file.
Image

Topic author
john71
Posts: 1009
Joined: 10.08.2016
With us: 8 years 4 months

Post #23by john71 » 10.11.2021, 22:11

fyr02 wrote:I'm saying that in the case of this texture, it is not justifiable to upscale the Neptune texture to 16K.

It is not true either.

When you are getting closer to Neptune in Celestia, the 16k texture gives you a much more realistic feeling. 4k is not enough, it seems blurry.

Added after 4 minutes 4 seconds:
fyr02 wrote:Such miniscule improvements (which Cupscale does such as keeping edges, reducing noise, which is the point of the ai upscale) are not representative of the real texture, and as such upscaling the texture past 4-6K is useless.

Take a look at the upscaled picture above. Cupscale is doing MUCH MORE. It recreates the image at a higher resolution.

Added after 9 hours 24 minutes:
fyr02 wrote:Upscaling your textures (such as in this case, where 2K was upscaled to 16K) will not improve the quality past a certain point. all that is being done is increasing the weight of the file.

I beg to differ.

Cupscale creates artificial mathematical details. Of course they are not "real" details, but they are NEW details. The new texture has a lot more information, so it adds a new aesthetic layer to the planet.

If you don't like this new, artificial aesthetic layer, use a 4k texture.

In my opinion the new artificial details make the planet more realistic, even if these details are just mathematical constructs.

Avatar
fyr02
Posts: 249
Joined: 19.04.2019
Age: 19
With us: 5 years 7 months
Location: San Diego, California (United States)

Post #24by fyr02 » 13.11.2021, 04:15

john71 wrote:Take a look at the upscaled picture above. Cupscale is doing MUCH MORE. It recreates the image at a higher resolution

This is exactly what I am saying. The edge-detection algorithims that help to keep the texture clean at higher resolutions suffers from overfitting at this scale. Nearly all of the clouds in the upscaled version has dark outlines and artifacts around the clouds - this is a result of the edge-detection algorithm, which has overfit the data and included the anti-aliasing in the final result.

This is very clearly not good - take this from someone who has worked with denoising and upscaling algorithms. It is essential to not overfit the data.
You can upscale it to 4K, but no more. otherwise the data will be overfit. Cupscale has its flaws, like any other algorithms, and as such it's important not to overdo something like this.

overfitting.png
Image

Topic author
john71
Posts: 1009
Joined: 10.08.2016
With us: 8 years 4 months

Post #25by john71 » 13.11.2021, 07:01

fyr02 wrote:Cupscale has its flaws, like any other algorithms, and as such it's important not to overdo something like this.

You seem to forget the following facts:

1.) 90+% of the upscaled image has no problems at all, like no artifacts

2.) this is the RAW output of Cupscale, the program can be fine tuned

3.) those minimal problems can be fixed by manual editing in Gimp (for example using noise reduction algorithms before the conversion)

4.) the overall aesthetic result is much better than the very blurry 4k base texture.

Avatar
trappistplanets
Posts: 639
Joined: 11.06.2020
Age: 17
With us: 4 years 6 months

Post #26by trappistplanets » 13.11.2021, 12:54

i am sorry, but i haft to agree with fyr
There is a limit to how far we can travel into the stars.
Owner of "The Grand Voyage" addon series
Collaborating with Dangerous_safety for the still developing Cer system

Topic author
john71
Posts: 1009
Joined: 10.08.2016
With us: 8 years 4 months

Post #27by john71 » 13.11.2021, 13:05

trappistplanets wrote:i am sorry, but i haft to agree with fyr

No problem, it is a matter of taste anyway.

Although I think the FACT that people are developing AI upscaling is in itself proves it is DIFFERENT than previous efforts and there is a demand for it. So I don't think it is worthless, so what are we talking about anyway?

Topic author
john71
Posts: 1009
Joined: 10.08.2016
With us: 8 years 4 months

Post #28by john71 » 25.11.2021, 12:28

"Cupscaled" 16k Saturn texture, from the 4k base image (Runar Thorvaldsen's Motherload file):

th_saturn.png-16x_BSRGAN.jpg

Topic author
john71
Posts: 1009
Joined: 10.08.2016
With us: 8 years 4 months

Post #29by john71 » 09.12.2021, 12:15

Standard Saturn Celestia installation texture upscaled to 16K:

saturn-4x_BSRGAN-realesrgan-x4plus.jpg


Added after 7 hours 31 minutes:
Standard Uranus Celestia installation texture upscaled to 16K:

uranus.jpg-realesrgan-x4plus-4x_BSRGAN-ed-ed.png-realesrgan-x4plus.jpg


Added after 21 minutes 5 seconds:
Standard Neptune Celestia installation texture upscaled to 16K (new edited):

neptune.jpg-realesrgan-x4plus ed ed.png-realesrgan-x4plus.jpg

Topic author
john71
Posts: 1009
Joined: 10.08.2016
With us: 8 years 4 months

Post #30by john71 » 10.12.2021, 16:12

I upscaled the NASA Pluto global texture from here:

https://solarsystem.nasa.gov/resources/818/pluto- ... ry=planets/dwarf-planets_pluto

Here is the result, 23k, 27MB (!!!).

pluto_color_mapmosaic.jpg-realesrgan-x4plus 23k.jpg
Last edited by john71 on 14.12.2021, 07:32, edited 1 time in total.

Avatar
trappistplanets
Posts: 639
Joined: 11.06.2020
Age: 17
With us: 4 years 6 months

Post #31by trappistplanets » 10.12.2021, 17:25

my computer does not like that thing
There is a limit to how far we can travel into the stars.
Owner of "The Grand Voyage" addon series
Collaborating with Dangerous_safety for the still developing Cer system

Topic author
john71
Posts: 1009
Joined: 10.08.2016
With us: 8 years 4 months

Post #32by john71 » 10.12.2021, 17:26

trappistplanets wrote:my computer does not like that thing

Open it in Gimp only.

Added after 5 minutes 54 seconds:
Trust me, it is SPECTACULAR when you zoom in!

Avatar
SevenSpheres
Moderator
Posts: 826
Joined: 08.10.2019
With us: 5 years 2 months

Post #33by SevenSpheres » 10.12.2021, 17:58

john71 wrote:Based on SevenSpheres Discord information, here is a Ryugu texture upscaled:

I uploaded it to Discord too, not stealing it! :smile:

Are you on the Discord server? What's your username there?

Edit: Yeah, I just saw your post. j-disc.
My Addons: viewtopic.php?f=23&t=19978 • Discord server admin
Celestia versions: 1.5.1, 1.6.1, 1.6.2, 1.7.0, and some unofficial versions like Celestia-ED

Topic author
john71
Posts: 1009
Joined: 10.08.2016
With us: 8 years 4 months

Post #34by john71 » 10.12.2021, 17:59

Ryugu 16k 23MB (!!!):
Last edited by john71 on 14.12.2021, 07:32, edited 1 time in total.

Avatar
trappistplanets
Posts: 639
Joined: 11.06.2020
Age: 17
With us: 4 years 6 months

Post #35by trappistplanets » 11.12.2021, 13:04

john71 wrote:Open it in Gimp only.
i can't even open the thing in a new tab to download it

Added after 1 hour 26 minutes:
wonder how this would look cubscaled up to 4k?
nix.jpg

nix texture i compiled myself
There is a limit to how far we can travel into the stars.
Owner of "The Grand Voyage" addon series
Collaborating with Dangerous_safety for the still developing Cer system

Topic author
john71
Posts: 1009
Joined: 10.08.2016
With us: 8 years 4 months

Post #36by john71 » 11.12.2021, 18:32

Here is the 8k version, but the base jpg file unfortunately has many artifacts.

nix ed.png-4x_RealSR_DF2K_JPEG.jpg

Avatar
trappistplanets
Posts: 639
Joined: 11.06.2020
Age: 17
With us: 4 years 6 months

Post #37by trappistplanets » 11.12.2021, 21:55

john71 wrote:Here is the 8k version, but the base jpg file unfortunately has many artifacts.
yeah i know....the nix image i used to cover the northern parts had issues
There is a limit to how far we can travel into the stars.
Owner of "The Grand Voyage" addon series
Collaborating with Dangerous_safety for the still developing Cer system


Return to “Ideas & News”