[WIP] The SH5 EcoMod

[Seaowl]

The "National Library of Scotland" has a very nice collection of maps (not only from scotland).


Here is the link:


https://maps.nls.uk/index.html

[Jeff-Groves]

What would be really nice is digital information on heights!

[hauangua]

Originally Posted by Jeff-Groves

What would be really nice is digital information on heights!

Maybe I wrong... But can help this? :

http://en-us.topographic-map.com/pla...Dover-9575707/

[bstanko6]

Nice work. Works great. The 2x 3x trees slows things down a bit, so I don't use them. Good work.

[OldCoder]

Jeff, this may be your answer:

https://github.com/Jorl17/open-eleva...er/docs/api.md

It's a public/free API for elevation queries. Examples are there to call it from curl, but obviously you could easily make a Python script/whatever to send queries.

For instance, in a web browser:

https://api.open-elevation.com/api/v...23826,1.317397

Response:
{"results": [{"latitude": 51.138584, "elevation": 105, "longitude": 1.366985}, {"latitude": 51.123826, "elevation": 7, "longitude": 1.317397}]}

The first set of coordinates are the cliffs to the eastern side of Dover port, returning a height of 105 metres.

The second set of coordinates are also Dover near the sands/seafront shopping area, returning 7m.

The API does respond quite slowly unfortunately (at least for me), so that might be a limiting factor.

[OldCoder]

Also Jeff, to solve the performance issue it should be possible to self-host the API (and data if you have large hard drives), which you probably already realised it being on github. this issue issue related to the above is worth a read. https://github.com/Jorl17/open-elevation/issues/27. It also mentions higher resolution map data at https://www.eorc.jaxa.jp/ALOS/en/aw3d30/index.htm

[Jeff-Groves]

Thanks OldCoder


Some interesting information in the links. I'll do more research into them.

What I did find in some formats is that the scale is wrong for SH5.
The z or up/down is fine but the x y is wrong. I think it could be averaged for our use.
Say we needed 5 units from a data set to make one unit for SH5. We could just average that and be close.

[Jeff-Groves]

So far I see the DSM .asc format as the most promising to work with.
I'll admit I have not followed it up fully.

[gap]

Originally Posted by Seaowl

WOW!! Fantastic! Great work.
So now it´s possible to import some Lighthouserocks directly into the map.
And this lighthouses will be desperatly needed, because the shelves will become real too...

That might depend on the size of each rock. SH5's height maps have a resolution of one point every fifty meters so, for little single rocks whose diameter is lesser than a few hundred meters, it might be more convenient resorting to external 3D meshes set as terrain objects. As for sandbanks, shoals, (semi-)submerged reefs, and other seabed natural features, indeed we can realistically model them directly in the terrain, as long as we have real bathymetric charts available.

Originally Posted by Seaowl

Is there more or less the same map in SH3-4-5?

So that everything, that would be done for one part, could be used for the others too?

Yes and no.

I think the base terrain files are common to all the SH games. Demonstration of that is the fact that, as tested by Jeff, the tool used by the GWX team for editing terrain files also works with SH5. The resolution of those files is extremely low though, with elevation points set 200 meters apart. As far as I could understand from Pintea's detailed explanations, SH5 uses two methods for increasing that resolution:

• in general areas, the terrain resolution is increased in memory to one point every 50 meters, and noise is applied to make the 'fake' heigh points not to look flat.
  
  
• In areas of bigger interest, Terrain Editor can be used to create 'real' height changes. The editor saves the new terrain elevation values as RAW data files, which are exactly the ones I and Jeff have been working on. Each RAW file covers an area of 5 x 5 sq. km, and again it has a resolution of one height point every 50 meters. RAW files work as terrain patches: i.e. when the game finds those files for areas within rendering range from the camera, low -detail height data from base terrain files gets overwritten with data from the said RAW files.

Unfortunately SHIII (and probably SHIV) has nothing like that, so yes, in theory we could convert the detailed height data from RAW files into greyscale heightmaps, as used by SH5's predecessors, but due to their poor resolution, they wouldn't look not nearly as good as in SH5.

Originally Posted by OldCoder

gap, is the en-gb.topographic-map.com site any use? For instance if you go there and type in Dover, you can see the height map for the coast line. I used to have printed nautical maps for the Ramsgate/Dover area showing height maps under the water but sadly I don't have those any more.

Originally Posted by hauangua

Maybe I wrong... But can help this? :

http://en-us.topographic-map.com/pla...Dover-9575707/

Thank you guys, I knew that website. Their height maps look good, but they have a problem: they are colored, which makes the grabbing of their information more complicated for us (greyscale i.e. 8-bit maps are much more easy to handle, though they have a poorer vertical resolution), and they are overlapped with location names, road symbols etc. which make them unusable for our purposes, unless we find a way to toggle them off (which I didn't find so far).

NASA maps are much easier to use though not all the planet is covered and seabed depth information is not provided by them:

https://gdex.cr.usgs.gov/gdex/

Originally Posted by Seaowl

The "National Library of Scotland" has a very nice collection of maps (not only from scotland).


Here is the link:


https://maps.nls.uk/index.html

Very nice, thanks, I will have a close look into that website!

Originally Posted by bstanko6

Nice work. Works great. The 2x 3x trees slows things down a bit, so I don't use them. Good work.

Okay, good to know. The next release will have also a couple of optional mods (work of Pintea) that will make trees 25% bigger or 25% smaller than in main mod. You might use bigger trees (as in stock game), to compensate for their sparser density

Originally Posted by Jeff-Groves

What would be really nice is digital information on heights!

Originally Posted by OldCoder

Jeff, this may be your answer:

https://github.com/Jorl17/open-eleva...er/docs/api.md

It's a public/free API for elevation queries. Examples are there to call it from curl, but obviously you could easily make a Python script/whatever to send queries.

For instance, in a web browser:

https://api.open-elevation.com/api/v...23826,1.317397

Response:
{"results": [{"latitude": 51.138584, "elevation": 105, "longitude": 1.366985}, {"latitude": 51.123826, "elevation": 7, "longitude": 1.317397}]}

The first set of coordinates are the cliffs to the eastern side of Dover port, returning a height of 105 metres.

The second set of coordinates are also Dover near the sands/seafront shopping area, returning 7m.

The API does respond quite slowly unfortunately (at least for me), so that might be a limiting factor.

Originally Posted by OldCoder

Also Jeff, to solve the performance issue it should be possible to self-host the API (and data if you have large hard drives), which you probably already realised it being on github. this issue issue related to the above is worth a read. https://github.com/Jorl17/open-elevation/issues/27. It also mentions higher resolution map data at https://www.eorc.jaxa.jp/ALOS/en/aw3d30/index.htm

That's very nice, though for me the API is so slow that I got a timeout error most of the times.

I say sorry if my questions might sound a bit silly but can I change coordinates values in web browser-based query with whatever world coordinate I want? Where are those height values obtained from?

Originally Posted by Jeff-Groves

Thanks OldCoder


Some interesting information in the links. I'll do more research into them.

What I did find in some formats is that the scale is wrong for SH5.
The z or up/down is fine but the x y is wrong. I think it could be averaged for our use.
Say we needed 5 units from a data set to make one unit for SH5. We could just average that and be close.

I don't quite get you here Jeff

[Seaowl]

As you mentioned height points every 50 meters:
On this site I´ve found some 50m Digitalterrainmodels (also available in 25m, 1“ and 3“)

http://data.opendataportal.at/dataset/dtm-norway

I downloaded Denmark and Norway.
Into the folders, I found .prj, .tfw and .tif files.
The .tif files I was able to open with Gimp: They contain black and white pictures without inscriptions.
Unfortunatly no seabed data (mainland is white, sea is black)
Don´t know, if this is of any use for you?

As far as I´ve seen, there is a difference between digital terrain and digital surface models. The latter ones map the landscape including vehicles and buildings.

That might depend on the size of each rock. SH5's height maps have a resolution of one point every fifty meters so, for little single rocks whose diameter is lesser than a few hundred meters, it might be more convenient resorting to external 3D meshes set as terrain objects. As for sandbanks, shoals, (semi-)submerged reefs, and other seabed natural features, indeed we can realistically model them directly in the terrain, as long as we have real bathymetric charts available.

Ok.
Hence for lighthouses such as Fastnet lighthouse,
it would be the best solution to make a 3D rock model
like the one you already did for Kendras lighthouse in SH3.

[gap]

Originally Posted by Seaowl

As you mentioned height points every 50 meters:
On this site I´ve found some 50m Digitalterrainmodels (also available in 25m, 1“ and 3“)

http://data.opendataportal.at/dataset/dtm-norway

I downloaded Denmark and Norway.
Into the folders, I found .prj, .tfw and .tif files.
The .tif files I was able to open with Gimp: They contain black and white pictures without inscriptions.
Unfortunatly no seabed data (mainland is white, sea is black)
Don´t know, if this is of any use for you?

As far as I´ve seen, there is a difference between digital terrain and digital surface models. The latter ones map the landscape including vehicles and buildings.

Thank you Seaowl

I have had a look into the height maps provided by opendataportal.at, and as far as I could understand its owner has made a great work in collecting and putting together world height data from many sources; I see two problems with them though:

• The world of all the SH games is based on the equirectangular map projection
  
  
   

  According to the definition given by Wikipedia, this projection "maps meridians to vertical straight lines of constant spacing (for meridional intervals of constant spacing), and circles of latitude to horizontal straight lines of constant spacing (for constant intervals of parallels)". Due to its distorsions, this kind of map projection is not used in marine cartography, but (quoting again Wikipedia) "because of the particularly simple relationship between the position of an image pixel on the map and its corresponding geographic location on Earth", it is quite popular in thematic mapping and commonly used in Gis technology and... submarine simulations lol.
  For small areas it is possible for us to empirically convert other map projections to an equirectangular projection, as far as the original projection is a cylindrical projection. The below charts are an example of that:
  
  http://map.openseamap.org/
  http://fishing-app.gpsnauticalcharts...54.1996/7.8663
  
  Unfortunately, the height maps offered by the website you pointed out use another type of projection which maps parallels and meridians as curved lines and which is not easy to overlap to the SH world.
  
  
• There are several free 3D editing programs capable of converting from 8-bit (greyscale) height maps to mesh coordinates (that we will be able to import in game when Jeff's importer/exporter will be ready) and vice-versa. Moreover greyscale maps can be easily edited in photoshop or in any other raster image editor though, having only 256 color shades for representing heights from pure black to pure white, their resolution is quite poor. Conversely, the height maps available at opendataportal.at adopt the 32-bit floating point deflate GeoTIFF format, capable of storing a much wider height range at excellent resolution, but not supported neither by photoshop nor by 3D editing programs (that's why when you open them you see only a B/W map). In order to extract their height data we would need another exporter, similar to the one Jeff is already working on.

Originally Posted by Seaowl

Ok.
Hence for lighthouses such as Fastnet lighthouse,
it would be the best solution to make a 3D rock model
like the one you already did for Kendras lighthouse in SH3.

Exactly!

[Jeff-Groves]

Originally Posted by gap

I don't quite get you here Jeff

Say we have height information at a resolution of 1 m.
Say SH5 needs 25m resolution.
Take a 5mx5m (25m) area from the 1m information and just average that.
All you do is add all the values and divide by 25.
You'll get the average height value for that area.

It's like getting your average in bowling. LOL!

[gap]

Originally Posted by Jeff-Groves

Say we have height information at a resolution of 1 m.
Say SH5 needs 25m resolution.
Take a 5mx5m (25m) area from the 1m information and just average that.
All you do is add all the values and divide by 25.
You'll get the average height value for that area.

It's like getting your average in bowling. LOL!

Yep, I understand that but so far I thought that SH5 needs 25m resolution and... we have height information at a resolution of 25 m lol

[Jeff-Groves]

Originally Posted by gap

Yep, I understand that but so far I thought that SH5 needs 25m resolution and... we have height information at a resolution of 25 m lol

yes. But I see some information set at 2m, 30m, and so on depending where you get the info. Still learning how all this works and studying file formats.
Just as the info we got to do the RAW stuff to this point?
The more info on different formats I learn could lead to a better RAW creator/editer.

[Jeff-Groves]

OBJ2RAW script files sent.


It could be better but I'd rather work on the stand alone version as not many want to pony up for 010.