NightLight: 64k VT earth night texture
-
Topic authornardo
- Posts: 35
- Joined: 25.08.2016
- Age: 27
- With us: 8 years 3 months
- Location: New Haven, CT, USA
- Contact:
NightLight: 64k VT earth night texture
i've edited the 2012 VIIRS map to appear dimmer, blacker, and more realistic, then tiled it with 2k tiles. below, i've put the full add-on along with screenshots and a 4k preview. i really liked the look so i've decided to finalize and share.
this is my first add-on, so please give suggestions. if you need it in a different tile size, i'll be happy to upload another version.
edit: now available as dds as well as jpg
edit: dds version now available in 1k tiles as well as 2k tiles
this is my first add-on, so please give suggestions. if you need it in a different tile size, i'll be happy to upload another version.
edit: now available as dds as well as jpg
edit: dds version now available in 1k tiles as well as 2k tiles
- Attachments
-
- NightLight64k_1k_dds.zip
- full 1k-tile dds add-on with ssc and readme
- (54.05 MiB) Downloaded 411 times
-
- NightLightDDS.zip
- full dds add-on with ssc and readme
- (53.99 MiB) Downloaded 380 times
-
- NightLight64k_2k.zip
- full jpg add-on with ssc and readme
- (55.58 MiB) Downloaded 695 times
Last edited by nardo on 16.09.2016, 23:53, edited 2 times in total.
PC: Acer Aspire E 15, Intel Core i5-6200U 2.3GHz (2.8GHz with Turbo Boost), 16GB DDR4, NVIDIA GeForce 940MX with 2GB VRAM, Windows 10 Home
- FarGetaNik
- Posts: 484
- Joined: 05.06.2012
- With us: 12 years 5 months
- Location: Germany
Good one, thanks for sharing! I also found the night texture to be too bright, just compare it to theese images of Rosetta:
http://www.esa.int/Our_Activities/Operations/Cities_at_night_Extraordinary_Rosetta_images
Night lights would probably be invisible to the naked eye. They become visible only on long-exposure photographs. I also never got why to display a blue surface, I mean the ground doesn't glow right? Your version is a lot more realistic. It might be still too bright, but they should remain visible, so it's just fine. I'm not sure if the lights are too yellowish, but there might not even be true-color data. Where did you get the data from? I only found a 16k texture online.
Right and... virtual textures are incompatible with cloud shadows but that's a different issue
http://www.esa.int/Our_Activities/Operations/Cities_at_night_Extraordinary_Rosetta_images
Night lights would probably be invisible to the naked eye. They become visible only on long-exposure photographs. I also never got why to display a blue surface, I mean the ground doesn't glow right? Your version is a lot more realistic. It might be still too bright, but they should remain visible, so it's just fine. I'm not sure if the lights are too yellowish, but there might not even be true-color data. Where did you get the data from? I only found a 16k texture online.
Right and... virtual textures are incompatible with cloud shadows but that's a different issue
Hi nardo.
Could you please upload a single level6 2K image of a well know place, e.g. a big city, in order to evaluate the correspondence with space taken images?
At a first glance, images look a bit blurry and excessively dim, in my opinion.
Thank you.
Bye
Goofy
Could you please upload a single level6 2K image of a well know place, e.g. a big city, in order to evaluate the correspondence with space taken images?
At a first glance, images look a bit blurry and excessively dim, in my opinion.
Thank you.
Bye
Goofy
"Something is always better than nothing!"
HP Omen 15-DC1040nl- Intel® Core i7 9750H, 2.6/4.5 GHz- 1TB PCIe NVMe M.2 SSD+ 1TB SATA 6 SSD- 32GB SDRAM DDR4 2666 MHz- Nvidia GeForce GTX 1660 Ti 6 GB-WIN 11 PRO
HP Omen 15-DC1040nl- Intel® Core i7 9750H, 2.6/4.5 GHz- 1TB PCIe NVMe M.2 SSD+ 1TB SATA 6 SSD- 32GB SDRAM DDR4 2666 MHz- Nvidia GeForce GTX 1660 Ti 6 GB-WIN 11 PRO
-
Topic authornardo
- Posts: 35
- Joined: 25.08.2016
- Age: 27
- With us: 8 years 3 months
- Location: New Haven, CT, USA
- Contact:
FarGetaNik
thank you! i got the full (54k x 27k) image by putting together the tiles here: http://earthobservatory.nasa.gov/Features/NightLights/page3.php
then, i tried to match observations from the ISS since that's what i was most interested in. the readme shows the steps i took. however, my computer can barely hold half of the full res image in gimp so i simplified the process by making it grayscale first then adding the hue later. this might be why it seems too yellow. still, i think it's pretty close to the true color images. they're definitely brighter than life, but as you said, they should stay visible. besides, that's what ctrl+l is for.
as for the cloud shadows, is that an issue on the night side of earth?
Added after 24 minutes 29 seconds:
Goofy
sure thing. here's a couple from the highest level. the blurriness is either exposure or the fact that i'm making a 64k image out of a 54k one. the dimness is intentional but i have a brighter version saved just in case.
thank you! i got the full (54k x 27k) image by putting together the tiles here: http://earthobservatory.nasa.gov/Features/NightLights/page3.php
then, i tried to match observations from the ISS since that's what i was most interested in. the readme shows the steps i took. however, my computer can barely hold half of the full res image in gimp so i simplified the process by making it grayscale first then adding the hue later. this might be why it seems too yellow. still, i think it's pretty close to the true color images. they're definitely brighter than life, but as you said, they should stay visible. besides, that's what ctrl+l is for.
as for the cloud shadows, is that an issue on the night side of earth?
Added after 24 minutes 29 seconds:
Goofy
sure thing. here's a couple from the highest level. the blurriness is either exposure or the fact that i'm making a 64k image out of a 54k one. the dimness is intentional but i have a brighter version saved just in case.
PC: Acer Aspire E 15, Intel Core i5-6200U 2.3GHz (2.8GHz with Turbo Boost), 16GB DDR4, NVIDIA GeForce 940MX with 2GB VRAM, Windows 10 Home
- FarGetaNik
- Posts: 484
- Joined: 05.06.2012
- With us: 12 years 5 months
- Location: Germany
Thanks for the link! Yeah my laptop also can barely handle such huge textures, my limit is somewhere at 20-25k, then each step takes about an hour to process and Gimp will likely crash.
So that's what it is good for? makes sense, but I still don't like exaggerated brightness.nardo wrote:besides, that's what ctrl+l is for.
No but when using a virtual texture Celestia completely refuses to render cloud shadows, so when using your addon they're gone for good. But I fixed this by making your texture an alternative surface, so I can switch back and forth when I need the one or othernardo wrote:as for the cloud shadows, is that an issue on the night side of earth?
- FarGetaNik
- Posts: 484
- Joined: 05.06.2012
- With us: 12 years 5 months
- Location: Germany
-
Topic authornardo
- Posts: 35
- Joined: 25.08.2016
- Age: 27
- With us: 8 years 3 months
- Location: New Haven, CT, USA
- Contact:
i meant the cloud shadows always freak out. my laptop can handle up to a 32k square in gimp, so i can process a 64k image in two parts. just verrrryy slowwlllyyy. anything else i have to do from the command line. it's a step up from my last laptop which could barely even open an 8k.
PC: Acer Aspire E 15, Intel Core i5-6200U 2.3GHz (2.8GHz with Turbo Boost), 16GB DDR4, NVIDIA GeForce 940MX with 2GB VRAM, Windows 10 Home
- FarGetaNik
- Posts: 484
- Joined: 05.06.2012
- With us: 12 years 5 months
- Location: Germany
Why is that? Don't you like the effect or are they also causing bugs? I experience black planet surfaces because of them sometimes.nardo wrote:i meant the cloud shadows always freak out.
I meant if all your textures are VT, how do you split them up into tiles? The ftex tool? It didn't work for me, that's why I'm asking. Doing that manually would be insane.
Then you have a better computer than me. I could not even load the texture you posted a link to... I hope the new version of gimp keeps its promise and works much faster. But I will need a new computer to really think about processing such huge textures.nardo wrote:my laptop can handle up to a 32k square in gimp, so i can process a 64k image in two parts. just verrrryy slowwlllyyy. anything else i have to do from the command line
-
Topic authornardo
- Posts: 35
- Joined: 25.08.2016
- Age: 27
- With us: 8 years 3 months
- Location: New Haven, CT, USA
- Contact:
how do i tile them? there's an old shell script for imagemagick by t00fri called virtualtex that tiles everything for me. i just have to make the textures and resize them for each level, then it takes care of the rest. it was a pain getting cygwin to work on my laptop and from what i've read it's an outdated method but it works fine for me.
(thank you John VV for uploading it to celestialmatters)
i just type "NightLight32k.png 2048 jpg" and boom, hundreds of tiles pour out.
Code: Select all
#! /bin/bash
if [ $# -lt 3 -o "$1" = "--help" ]; then
echo
echo 'Usage: virtualtex [--help | <texture name> <tile size> <tile format>] [e|E|w|W]'
echo
echo
echo The shell script \'virtualtex\' is a tool for Celestia \> 1.3.0
echo that supports \'virtual textures\'.
echo
echo The script generates the required tiles tx_i_j of \
desired \(square\) size,
echo \<tile size\>, in a specified format, \<tile format\> = \
png, tga, jpg,...,
echo from an input texture, \<texture name\>, in any popular graphics format.
echo
echo The optional 4th argument e\|E\|w\|W is for the case of tiling
echo square /e/astern \| /w/estern halfes of the full texture separately!
echo
echo Besides Linux/Unix, the script also runs in a current Cygwin \
installation
echo under Windows, \( http://www.cygwin.com \). If the z-shell \(\'zsh\'\) \
is unavailable,
echo it also may be executed with the \'bash\' shell, by replacing \
\#\! /usr/bin/zsh by
echo \#\! /bin/bash in the first line.
echo
echo The script assumes that a recent version \(\>= 6.1.8\) of the ImageMagick package
echo \( http://www.ImageMagick.org \) is installed \
\(either under Unix/Linux or Windows\).
echo The utilities \'convert\' and \'identify\' of that package are used.
echo
echo You may increase the pixel cache size \$maxmem from the 80 MB default
echo value to e.g. 80\% of your RAM size within the script with an editor.
echo This will speed up the performance of \'virtualtex\' considerably.
echo
echo On a PIII/512MB RAM the tiling of a 16k x 8k texture into
echo 32 \(2k x 2k\) tiles now only takes 15 minutes with a Linux OS!
echo
echo Author: Dr. Fridger Schrempp, fridger.schrempp@desy.de
echo Version: 1.04, 02/15/05
echo
else
maxmem=80
texturesize=`identify -format "%wx%h" $1`
texturewidth=`identify -format "%w" $1`
textureheight=`identify -format "%h" $1`
tilesize=$2
tileformat="$3"
echo
echo "Texture size = " $texturewidth "x" $textureheight "tilesize = " $tilesize
echo "Number of tiles =" $(( ((texturewidth/tilesize)) * ((textureheight/tilesize)) ))
echo "Image format of tiles:" $tileformat
echo
echo "Tile: "
echo
convert -cache $maxmem -crop ${tilesize}x${tilesize} $1 out%d.$tileformat
j=0
while (( j * tilesize + tilesize <= textureheight )); do
((offy = j * tilesize))
if [ $# -eq 4 ]; then
if [ "$4" = "e" -o "$4" = "E" ]; then
ioff=$(( texturewidth/tilesize ))
elif [ "$4" = "w" -o "$4" = "W" ]; then
ioff=0
else
echo
echo "*** Incorrect 4th parameter! ***"
echo
return
fi
fi
i=0
while (( i * tilesize + tilesize <= texturewidth )); do
((offx = i * tilesize))
echo "tx_"$((i + ioff))"_"${j}": x-offset:" $((offx + ioff * tilesize)) "y-offset:" $offy
mv out$((i+ ((texturewidth/tilesize)) * j)).$tileformat tx_$((i + ioff))_${j}.$tileformat
((i++))
done
((j++))
done
fi
(thank you John VV for uploading it to celestialmatters)
i just type "NightLight32k.png 2048 jpg" and boom, hundreds of tiles pour out.
PC: Acer Aspire E 15, Intel Core i5-6200U 2.3GHz (2.8GHz with Turbo Boost), 16GB DDR4, NVIDIA GeForce 940MX with 2GB VRAM, Windows 10 Home
- FarGetaNik
- Posts: 484
- Joined: 05.06.2012
- With us: 12 years 5 months
- Location: Germany
Hi nardo.
In my life I travelled by air for almost one million miles, so I had the possibility to see a lot of times lighted nightscapes.
I can say for sure that, if there are no clouds and no haze, fog or whatelse between your aircraft and the ground, what you can see is absolutely closer to the right image than to the left one.
I modified only contrast and luminosity, nothing else.
BTW, reducing such a way luminosity and contrast, you killed resolutiom feeling too, so that it looks like a smaller image.
Just my opinion, obviously.
My little cent.
Click on the image to see it full size.
Bye
Goofy
Added after 34 minutes 31 seconds:
Hi nardo.
I just downloaded your zip, to clarify a doubt I had looking at its very small dimensions (only 56MB for a 64K VT, while we are used to hundred MBs size of Motherlode's VTs!).
So I found the reason of this: it's up to Level4 instead of Level6, so really the VT is a 16K, not a 64K.
Am I missing something?
Thank you.
Goofy
In my life I travelled by air for almost one million miles, so I had the possibility to see a lot of times lighted nightscapes.
I can say for sure that, if there are no clouds and no haze, fog or whatelse between your aircraft and the ground, what you can see is absolutely closer to the right image than to the left one.
I modified only contrast and luminosity, nothing else.
BTW, reducing such a way luminosity and contrast, you killed resolutiom feeling too, so that it looks like a smaller image.
Just my opinion, obviously.
My little cent.
Click on the image to see it full size.
Bye
Goofy
Added after 34 minutes 31 seconds:
Hi nardo.
I just downloaded your zip, to clarify a doubt I had looking at its very small dimensions (only 56MB for a 64K VT, while we are used to hundred MBs size of Motherlode's VTs!).
So I found the reason of this: it's up to Level4 instead of Level6, so really the VT is a 16K, not a 64K.
Am I missing something?
Thank you.
Goofy
"Something is always better than nothing!"
HP Omen 15-DC1040nl- Intel® Core i7 9750H, 2.6/4.5 GHz- 1TB PCIe NVMe M.2 SSD+ 1TB SATA 6 SSD- 32GB SDRAM DDR4 2666 MHz- Nvidia GeForce GTX 1660 Ti 6 GB-WIN 11 PRO
HP Omen 15-DC1040nl- Intel® Core i7 9750H, 2.6/4.5 GHz- 1TB PCIe NVMe M.2 SSD+ 1TB SATA 6 SSD- 32GB SDRAM DDR4 2666 MHz- Nvidia GeForce GTX 1660 Ti 6 GB-WIN 11 PRO
- FarGetaNik
- Posts: 484
- Joined: 05.06.2012
- With us: 12 years 5 months
- Location: Germany
Goofy wrote:So I found the reason of this: it's up to Level4 instead of Level6, so really the VT is a 16K, not a 64K.
Am I missing something?
I checked and found the resoluton WAY better than the 8k night texture I use (the 16k versions lets Celestia crash) so it can't be just a 16k. I checked and found the tile size to be 2048, so level0 is already 4k. So 1 is 8k, 2 is 16k, 3 is 32k and 4 is 64k. I guess the low file size is due to the fact that it is jpg and the large black areas are easy to compress. That lets me wondering if png would be even more effective compressing the vast oceans... or you could just through out every tile that contains nothing but black, Celestia doesn't care, I tried that with a specular map, it worked just fine.
But one more thing, I found tile size of 512 a bit unhandy, since you'll end up with thousands of tiles and the Celestia constantly has to load different ones. But 2048 pixel might be a bit too much, but it loads smoothly when a jpg as used here, so it's just fine. But a 2k tile in png might cause some performance problems maybe...
Added after 5 minutes 48 seconds:
Goofy wrote:In my life I travelled by air for almost one million miles, so I had the possibility to see a lot of times lighted nightscapes.
I can say for sure that, if there are no clouds and no haze, fog or whatelse between your aircraft and the ground, what you can see is absolutely closer to the right image than to the left one.
I modified only contrast and luminosity, nothing else.
BTW, reducing such a way luminosity and contrast, you killed resolutiom feeling too, so that it looks like a smaller image.
Just my opinion, obviously.
Ok that's your opinion, but let me explain my thoughts on that, don't take that personally. Our eyes are good in adaption brighness, so the city lights just appear bright at night. I think the link I posted proves the brightnes is nothing compared to the sunlit surface of the Earth. In the picture you posted it might seem blurrier, but that's most likely due to the contrast difference between the images. When in Celestia and adapting to the brighness of the lights, you might just get the same apparent sharpness. And obviosly stretching contrast doesn't add real detail in the image but only amplyfies the details already present.
-
Topic authornardo
- Posts: 35
- Joined: 25.08.2016
- Age: 27
- With us: 8 years 3 months
- Location: New Haven, CT, USA
- Contact:
Goofy
what FarGetaNik said. however, if you can tell me exactly how you made your version (ie which numbers you used for lum and contast), i'll give you the 64k texture for it. maybe we can call it NightBright.
FarGetaNik
i'll see if my laptop can make a 1k tile version and post it here. pretty sure that's the same tile size as Jestr's texture, which even my last laptop could handle. perhaps we can call it NightLite.
what FarGetaNik said. however, if you can tell me exactly how you made your version (ie which numbers you used for lum and contast), i'll give you the 64k texture for it. maybe we can call it NightBright.
FarGetaNik
i'll see if my laptop can make a 1k tile version and post it here. pretty sure that's the same tile size as Jestr's texture, which even my last laptop could handle. perhaps we can call it NightLite.
PC: Acer Aspire E 15, Intel Core i5-6200U 2.3GHz (2.8GHz with Turbo Boost), 16GB DDR4, NVIDIA GeForce 940MX with 2GB VRAM, Windows 10 Home
- FarGetaNik
- Posts: 484
- Joined: 05.06.2012
- With us: 12 years 5 months
- Location: Germany
nardo wrote:FarGetaNik
i'll see if my laptop can make a 1k tile version and post it here. pretty sure that's the same tile size as Jestr's texture, which even my last laptop could handle. perhaps we can call it NightLite.
Actually your version works surprisingly fine! It would be an issue if using png I guess.
- John Van Vliet
- Posts: 2944
- Joined: 28.08.2002
- With us: 22 years 2 months
i think you might find that using 512x512 tile sizes works best or 1024x1024
2k tiles are a bit big
also why start with the bluemarble jpg's that are way OVER exposed they look like the top 10% are set to 255
NOAA has the monthly data
the June 2016 "VCMSLCFG" data
http://ngdc.noaa.gov/eog/viirs/download_monthly.html
my home town area " Metro Detroit Area "
the viits data and the above jpg in the leve4 folder
long Island and NY.NY
now the monthly data needs a bit of work ( mainly near the poles ) and it is black and white 32 bit float tif's
2k tiles are a bit big
also why start with the bluemarble jpg's that are way OVER exposed they look like the top 10% are set to 255
NOAA has the monthly data
the June 2016 "VCMSLCFG" data
http://ngdc.noaa.gov/eog/viirs/download_monthly.html
my home town area " Metro Detroit Area "
the viits data and the above jpg in the leve4 folder
long Island and NY.NY
now the monthly data needs a bit of work ( mainly near the poles ) and it is black and white 32 bit float tif's
- FarGetaNik
- Posts: 484
- Joined: 05.06.2012
- With us: 12 years 5 months
- Location: Germany
John Van Vliet wrote:NOAA has the monthly data
the June 2016 "VCMSLCFG" data
http://ngdc.noaa.gov/eog/viirs/download_monthly.html
Wow they are huge! One part is more than a GB! Would be great if someone could process this.
I agree they seem a bit overexposed, but it's even worse with what I got to work with. And I knew this was black and white.
- John Van Vliet
- Posts: 2944
- Joined: 28.08.2002
- With us: 22 years 2 months
i did a few years agoWould be great if someone could process this.
http://forum.celestialmatters.org/viewtopic.php?f=9&t=606
http://forum.celestialmatters.org/viewtopic.php?f=9&t=451
the links are dead but i can reupload them
one is here
the 8 bit/Chanel red,green,blue colorized "64k.EarthLightsRGB.1.7.14.ppm.7z"
https://drive.google.com/file/d/0B6ZYAd08tZL-Wjg0a2pma2xUQmM/view?usp=sharing
Last edited by John Van Vliet on 14.09.2016, 20:36, edited 1 time in total.