[WIP] The SH5 EcoMod

[OldCoder]

gap,

I say sorry if my questions might sound a bit silly but can I change coordinates values in web browser-based query with whatever world coordinate I want? Where are those height values obtained from?

Yes you can. The height values are obtained from 250m SRTM data. I found links in the project github issues to download that data, and then it would just be a case of cloning the open-elevation code so that it would run on a local web server (or query direct via python). Then there would be almost zero latency in the query response time. Problem is 250m may or may not be accurate enough for your needs.

[gap]

Originally Posted by Jeff-Groves

OBJ2RAW script files sent.


It could be better but I'd rather work on the stand alone version as not many want to pony up for 010.

Got them and they work, very well done Jeff, you are the man!

Now it is part of my job putting your scripts at work

Originally Posted by Jeff-Groves

yes. But I see some information set at 2m, 30m, and so on depending where you get the info. Still learning how all this works and studying file formats.
Just as the info we got to do the RAW stuff to this point?
The more info on different formats I learn could lead to a better RAW creator/editer.

I perfectly got your point now

Real world height data with a resolution much lesser than 50 m would still be useful for general terrain profile, especially for the seabed whose visual aspect is not so important, and we could add some random noise to that data so to simulate a resolution to the resolution we have in game; anyway such a poor resolution would be of small use for recreating realistic coastal profiles and elevations, and if we used only that data we would end up with a smoothed out version of the world.

For height data with a resolution bigger than 50 m I see no problem: as you said we can average that data. Some manual adjustments might still be needed near the coasts for avoiding an undesirable "zigzags", but that kind of post-processing could be quickly performed in a 3D program (Wings3D 'sculpt' mode totally fits this need) or with the Terrain Editor that devs gave us.

Originally Posted by OldCoder

gap,

Yes you can. The height values are obtained from 250m SRTM data. I found links in the project github issues to download that data, and then it would just be a case of cloning the open-elevation code so that it would run on a local web server (or query direct via python). Then there would be almost zero latency in the query response time. Problem is 250m may or may not be accurate enough for your needs.

Okay, I thought that that set of world elevation values had a better resolution; I compared some of the heights it provides with Google Earth's data, and they seem largely comparable if not identical.

For the rest, see my considerations above please: indeed 250m is not the best, but it is still of some use. The problem is that I have no idea on how to create a local query like the one you have described

[gap]

@ OldCoder, Jeff-Groves and all the concerned ones

I have found another STRM database promising high resolution (30-m) world height data:

https://dwtkns.com/srtm30m/

From what I have read from STRM documentation available elsewhere, the resolution given is relative to the equator. Due to the distorsions of the SH world, the in-game resulting resolution will be slightly lesser than that, especially near the poles, but still bigger than the maximum terrain resolution of SH5 which is equal to 50m.

There are two problems with that set of data:

1 - some areas are not covered, and data provided seem only relative to landmasses (no sea depth data)

2 - height data is provided in SRTMHGT format; I don't know how easy would be for us processing that kind of files. Moreover, in order to download the satellite data an username and password are required, but I didn't see any way to register into the site. Greyscale previews of each tile are freely accessible. They cover an area of 1 deg of latitude x 1 deg of longitude and they have a size of 3601x3601 pixels; realtive to our game, that makes an horizontal resolution of 120,000 ÷ 3,600 = 33.33m, more than we need, but being 8-bit images they can store only 256 shades of gray/height levels. Depending on the height range stored in each tile, this might involve a quite poor vertical resolution and an overall lack of accuracy considering that each tile might use its own greyshade-to-height scale, and that those scales are not provided

[OldCoder]

Inspecting the URL for the download link, and trying the web site, it seems that registering an account at https://urs.earthdata.nasa.gov/ may be possible. It looks free to anyone, though I haven't tried it.

EOSDIS data are openly available to all and free of charge except where governed by international agreements.

[gap]

Originally Posted by OldCoder

Inspecting the URL for the download link, and trying the web site, it seems that registering an account at https://urs.earthdata.nasa.gov/ may be possible. It looks free to anyone, though I haven't tried it.

Thank you OldCoder, you are right!

I had already registered on https://urs.earthdata.nasa.gov/, and logging in into it allowed me to download the .hgt file relative to the German Bight tile. Now we need to understand how we can convert such a file format into something that can be easily imported in game

P.S: this link might be of some use:
https://github.com/zbycz/srtm-hgt-reader

[OldCoder]

That 30m data and the hgt reader at github was a good find.

I've cooked up a little c++ exe that will read the hgt and dump the contents into output files. One output file is text format, and looks like:

51.999722 1.000000 47
51.999722 1.000833 47
51.999722 1.001666 46

where that's LAT/LON and height in metres.

Second format is a greyscale image in PGM format, documented here: https://en.wikipedia.org/wiki/Netpbm_format.

The image created looks exactly like the preview button image at the earthdata site, so I know it's correct (at least for the single tile I tested).

So, download the height.zip file and put it into an empty folder.
Then download a srtm hgt tile file and drop it into an srtm subfolder.

Then run it like:

height.exe 51 1 3 200


51 1 are the LAT/LON from the tile filename.
3rd param is the stepping value. Specifying 3 results in a 1201x1201 image file. Specifying 1 is the highest resolution of 30m 1 arc-second data, and generates output at 3601x3601.
The 4th param is calibration of black to white, mapped to max height. So if you specify 200, then 0 is black and anything at 200+m will be drawn (and clipped if necessary) in white. You could specify 1000 instead and then a lot of low lying ground will be black or very dark, and only the highest mountains would be white.

The text file output scales similarly in size according to the 3rd param.


Also, you can run the exe with only 2 params just to return height at a specific LAT/LON if you want:

height.exe 51.389057 1.384060 and it will return interpolated height of 9.3m


I've only tested it briefly, so it may be a bit buggy lol.


Unfortunately the forum won't let me upload the exe zip to this message, but when I find out a way to upload it I will do so.

[Jeff-Groves]

Nice link to github.
May compile that in VS 2008 and see what needs done to adapt for RAW.

Oh OldCoder beat me to it! LOL!
I can send you my e-mail address and you can send it to me.

[Jeff-Groves]

Originally Posted by OldCoder

Also, you can run the exe with only 2 params just to return height at a specific LAT/LON if you want:

height.exe 51.389057 1.384060 and it will return interpolated height of 9.3m

This part interests me.
Say we build a matrix to hold just the height data for a given grid that matches what we need in SH5 then save that info as a float for each height. Output a proper format and you have a RAW file.

We can edit the RAW files now in a 3D program and convert back to RAW.
I butchered up some scripts for 010 that do the RAW to obj and back pretty well so far.
A stand alone program is in the works.

[gap]

Thank you OldCoder!

Jeff is right in suggesting to adapt the output of your executable so that the exported terrain height values cover exactly one of the SH5 RAW maps.

Each data tile provided in the 30-m STRM database is 1deg of longitude by 1deg of latitude in size, and it has a resolution of 1 arc-second. Conversely SH5 height maps measure 1/24 deg of lat/long, and they have a resolution of 1.5 arc-seconds.

Knowing that, I suppose that it shouldn't be too complicated modifying your code so that for each export session it will extract from the appropriate hgt file only the elevation values we need for one of the RAW maps, down-sampling them to the desired resolution. Ideally, the exe should accept three user imputs. The first two onputs would be numbers comprised between 0 and 23, indicating the position of the data array we want to extract relative to the big hgt tile, while the third parameter should determine the resolution of the exportd data (1.5 if omitted).

What do you think guys?

[OldCoder]

Hi, yes I just sent Jeff a link to the mediafile link for the exe.

If it works for Jeff I can easily upload sources so you guys can adapt as needed (or just do your own thing based on what's in github). The github code works (with a few fixups), and I then modified it to generate the output formats I was interested to see.

[gap]

Originally Posted by OldCoder

Hi, yes I just sent Jeff a link to the mediafile link for the exe.

If it works for Jeff I can easily upload sources so you guys can adapt as needed (or just do your own thing based on what's in github). The github code works (with a few fixups), and I then modified it to generate the output formats I was interested to see.

Loking forward to your and Jeff's news

You are amazing guys

[gap]

@ OldCoder

I am trying to run your tool as you said:

I created a new folder in C:\, I dropped your executable into it, I created a subfolder of the first folder, I named it 'strm' and I dropped there one of the hgt files, but when I try running height.exe from the command prompt I get a missing VCRUNTIME14D.dll error even though I have the latest Visual C++ 2017 runtimes installed (x86 and x64).

This happens in Windows 8.1. Is there anything I am doing wrong?

[OldCoder]

Try searching google/MS for Microsoft Visual C++ 2015 Redistributable

That should sort it (I think).

[Jeff-Groves]

OldCoder answered quicker then me.
And you need VCRUNTIME140.dll not VCRUNTIME14D.dll
Get the full install as there may be other dll's you need.

https://www.microsoft.com/en-us/down....aspx?id=52685

[gap]

Originally Posted by OldCoder

Try searching google/MS for Microsoft Visual C++ 2015 Redistributable

That should sort it (I think).

Originally Posted by Jeff-Groves

OldCoder answered quicker then me.
And you need VCRUNTIME140.dll not VCRUNTIME14D.dll
Get the full install as there may be other dll's you need.

https://www.microsoft.com/en-us/down....aspx?id=52685

The missing dll is actually VCRUNTIME140D.dll, sorry for mispelling its name, and unfortunately installing the Microsoft Visual C++ 2015 Redistributable didn't fix the problem



I am now trying to install the Microsoft Visual C++ 2015 Redistributable updates and to re-installe Microsoft Visual C++ 2017 Redistributables (I had to uninstall the latter in order to install the 2015 version)