SH4 Stopped Saving "SaveData.map" Data For Stellarium

[Capt Codetrucker]

Hello,


The title tells it all. I have no idea why, but it is impossible to use the "SHtoStellarium" script to copy my SH4 time and position to Stellarium.

The below spoiler contains some extensive information which may help.



Capt Codetrucker (Bullhide009)Yesterday at 8:47 PM
Well, I'm not sure what happened, but the script stopped working. I am getting this error...

Traceback (most recent call last):
File "C:\Users\cjw\Desktop\SH4toStellarium.py", line 37, in <module>
longitude = float(content[longst+4:longnd])/120000
ValueError: could not convert string to float: 'ark]\nPointsNb=0\n\n[Ruler]\nPointsNb=0\n\n[Range]\nPointsNb=0\n\n[Angle]\nPointsNb=0\n\n[Waypoint]\nPointsNb=0\n\n[KGSelector]\nPointsNb=0'

This happened when I performed a fresh install of SH3 and SH4.

Does this look familiar? Any ideas?
Capt Codetrucker (Bullhide009)Yesterday at 9:45 PM
FMI - Am I supposed to post technical requests here or on the Subsim forum?
October 5, 2020
Capt Codetrucker (Bullhide009)Today at 7:10 AM


SUCCESS! I have overcome the fails, but only by going back to the raw script in the doc. I never was able to ferret out the culprit code, but since my present customized script works I am going to enjoy it for awhile. Thanks for all the help.
Capt Codetrucker (Bullhide009)Today at 7:31 PM


FAILURE! After getting a number of successful SH4 to Stellarium exports, I am now having the same error as the 3rd post above this one. The error Traceback is the same, "ValueError: could not convert string to float:" I have determined the error comes from the fact SH4 is not producing a good "SaveData.map" file, but I have no idea why?
Any ideas?

UPDATE: Verified the failure is due to SH4 producing an erroneous SaveData.map file. There is no data in the sections of the file, so the script is attempting to read in only the structure of the file and not the data which should be there. For some reason I have been able to get SH4 to produce the needed data and then it fails. No rhyme or reason yet.





Thank you in advance for the assistance.

[propbeanie]

With some of the older SH tools, you had to have the utility in the Save folder for it to properly find and gather the needed data. I'm sure a custom script could point the app to the proper location, but is it using your script still?

[Pisces]

For debugging purpose I would add statements (or let the debugger show the contents of variables) that show what "content[longst+4:longnd]" or just longst and longnd contain. Something in this expression does not allow it to become a float. I'm too unfamiliar with Python to know what that could be, just familiar enough in general with such languages that it always helps to check the intermediate conversion steps.

[Capt Codetrucker]

@propbeanie - My laptop crashed several hours ago, so I am having to start from scratch. I was using the script and it produced as file, but since the correct numerical data was not produced the script threw errors because the data it did read was alphanumeric which the script could not digest.

@Pisces - That is a good idea. Thanks.




The problem is not the script. It works well if I feed it valid data manually. The culprit is SH4 itself. The SaveData.map files is not being populated. The framework is there, but no data, like my position is saved.



I will report back.

[propbeanie]

I have a SaveData.map in some of my "more involved" saves, and this is all it has:
[Mark]
PointsNb=0

[Ruler]
PointsNb=0

[Range]
PointsNb=0

[Angle]
PointsNb=0

[Waypoint]
PointsNb=2
Pt0=14383529.11,1699511.52,-2.97
Pt1=13184944.00,1465757.00,0.00

[KGSelector]
PointsNb=0
Near as I can tell, there is no "CurrentLocation", or whatever the game would call it. It does record the Waypoint locations, beginning the numbering at "zero". I'm not certain if that is long, lat, heading, or what for the Waypoints. Now, some of the Saves I have that do not have Waypoints or marks, etc., the SaveData.map is not present. I wonder if the py script has error catching for if the file does not exist yet? Also, since SH4 saves a lot of its data in txt files, the numerical characters would be string. Python does have a type-cast, doesn't it? Such that you can do a "string-to-float"?

[Pisces]

In that case, place a waypoint and save. :) Pt0 should be the location of the sub at time of the save.

[propbeanie]

:doh: I would say that you are 99.999999% correct Pisces (catch the SH4 6-digit joke??). I do not understand why that failed to register with me to begin with :doh: :O: - Most of my test Saves for FotRSU have one waypoint set, just to be certain the Load of Saves actually function correctly... :roll: :salute:

[Pisces]

Sorry, unfamiliar with 6 digits in the context of SH4.

I always set a waypoint as a stopgap for the helmsman that goes out for coffee when he figures he is close enough to the commanded course, but doesn't maintain it. At high timecompressio he will wander off course.

[propbeanie]

The "joke" part is that SH4 saves almost ~everything~ with six decimal places, whether it needs that much "resolution" or not. The Save folders are named with eight digits... ah well.

The bad helming was mentioned years ago by someone as a "simulation" of how difficult it is to navigate the boat. I don't know what that has to be so inaccurate, when you can shoot a torpedo with auto-targeting from 3500 yards, and it doesn't wander a whit away from the point of aim... :hmmm: - Again, ah well... it is a game... :arrgh!: - and it is still fun, over 13 years later! :salute:

[Capt Codetrucker]

@propbeanie
The sub's location is in there...


[Waypoint]
PointsNb=2
Pt0=14383529.11,1699511.52,-2.97
Pt1=13184944.00,1465757.00,0.00


The script points to "Pt0" and sends it to Stellarium. What I see in my SaveData.map is...


[Waypoint]
PointsNb=1
Pt0=0


...which is not a long enough string of numbers, so the script picks up all the structure which is garbage.

[propbeanie]

If you don't have a course set, the game seemingly assumes you are flying by the seat of your pants, and does not track waypoints. I do not know why it would NOT track where you are at the time, but... apparently it doesn't.

[Capt Codetrucker]

If you don't have a course set, the game seemingly assumes you are flying by the seat of your pants, and does not track waypoints. I do not know why it would NOT track where you are at the time, but... apparently it doesn't.


Are you saying you got the same...


[Waypoint]
PointsNb=1
Pt0=0


...if you saved a game without plotting a course? That is what it sounds like you mean, but I don't want to be presumptuous.


I think I will test a few scenarios.

[Pisces]

The "joke" part is that SH4 saves almost ~everything~ with six decimal places, whether it needs that much "resolution" or not. The Save folders are named with eight digits... ah well. I see, yes. I don't consider that a big deal. They better keep it simple and just write down the coordinates to the significant digits the float/decimal value can represent. No need for fancy code to cull insignificant digits not really important that could be the cause of more bugs. It's fine, you weren't supposed to see it anyway as a player.


The bad helming was mentioned years ago by someone as a "simulation" of how difficult it is to navigate the boat. I don't know what that has to be so inaccurate, when you can shoot a torpedo with auto-targeting from 3500 yards, and it doesn't wander a whit away from the point of aim... :hmmm: - Again, ah well... it is a game... :arrgh!: - and it is still fun, over 13 years later! :salute:3500 yards at practically real time is a big difference from careening your boat over 350nm in time compression.

[Capt Codetrucker]

Okay, I can't figure this out. Sometimes SH4 will produce usable "Pt0=" lat/lon and most of the time it doesn't. I have not yet figured out why sometimes :up: and sometimes :down:. I am going to think through different controlled procedures and methodically try them one by one.

[Capt Codetrucker]

UPDATE:


SH4 does produce last saved lat/lon of the sub. I do not know why this does not work in testing, but in sailing the sub not only can one save the last position, but overwriting a previous save works without a hitch.



The script works flawlessly if the script is initiated via a Python icon either as a direct saved icon or as a shortcut. For whatever reason, executing the script from the PyScripter IDE is unreliable. I do not know why.



The only item left to narrow down is the actual plotting of the marks on on the SH4 map from the sight reductions from Stellarium. The incongruity of the longitudes is proving to be causing niggles.

We press on! :)

[derstosstrupp]

If you use your DR as the assumed position and thereby keep the intercepts short, the longitude thing won’t be an issue. Problems arise when you use methods like HO 229 that require plotting from a whole degree of lat/long because the intercepts can be large.

If you do want to use larger intercepts, take the difference in minutes of longitude between assumed position and fix, and divide by the cosine of the latitude. Then shift the fix horizontally by the difference.

So say you plotted from 54 deg N, 2 deg E assumed position and got your fix at 53 deg 48’ N and 2 deg 13’ E. The difference in minutes is 13, so take 13 / cos (53.8) which gives 22. So your fix is 53 deg 48’ N, 2 deg 22’ E..

[Capt Codetrucker]

If you use your DR as the assumed position and thereby keep the intercepts short, the longitude thing won’t be an issue. Problems arise when you use methods like HO 229 that require plotting from a whole degree of lat/long because the intercepts can be large
...


Thanks. I have been taking my shots three-ish hours apart. Is that short enough?

[Pisces]

Celestial sights taken every 3 hours seems overkill. Consider how far you expect to move from the last place in that time? Dead reckoning should be the major method of progressing your position. Then a few times a day a celestial fix when possible to correct it. There is no sea current or wind drift in the game so you won't be too far off.

[derstosstrupp]

Thanks. I have been taking my shots three-ish hours apart. Is that short enough?

That’s not what I mean. So a key part of celestial navigation is deciding the point from which you plot your intercepts in order to determine lines of position. The distance between the point that you plot from, called the assumed position, and the point along the azimuth toward or away from the body that you plot the perpendicular line of position at is called the intercept. That intercept distance is what I’m talking about. Not distance between sights.

Also, as I mentioned on Discord, if you are doing celestial sights every three hours you will get burned out in no time. I think I suggested to you to keep it to one a day, that is more than sufficient given the fact that, as Pisces said, there’s no set and drift or anything like that in the game.

[Capt Codetrucker]

FAB Thanks guys.



FYI - I have been doing lots of shots as tests of SHtoStellarium. I have also begun to modify the script to use with FSX props and propliners. It's kind of tough since I also have to code in the altitude, but I digress. All in good time.

[Capt Codetrucker]

...
So say you plotted from 54 deg N, 2 deg E assumed position and got your fix at 53 deg 48’ N and 2 deg 13’ E. The difference in minutes is 13, so take 13 / cos (53.8) which gives 22. So your fix is 53 deg 48’ N, 2 deg 22’ E..

@ derstosstrupp...

Please examine the following to make sure I understand. ALL CAPS are for my comments. I am not yelling. :)

"So say you plotted [IN SILENT HUNTER] from 54 deg N, 2 deg E assumed position and got your fix at 53 deg 48’ N and 2 deg 13’ E [IN SILENT HUNTER]. The difference in minutes is 13, so take 13 / cos (53.8) which gives 22. So your fix [IN GOOGLE EARTH] is 53 deg 48’ N, 2 deg 22’ E."

So, do I understand correctly or am I backwards?

[derstosstrupp]

@ derstosstrupp...

Please examine the following to make sure I understand. ALL CAPS are for my comments. I am not yelling. :)

"So say you plotted [IN SILENT HUNTER] from 54 deg N, 2 deg E assumed position and got your fix at 53 deg 48’ N and 2 deg 13’ E [IN SILENT HUNTER]. The difference in minutes is 13, so take 13 / cos (53.8) which gives 22. So your fix [IN GOOGLE EARTH] is 53 deg 48’ N, 2 deg 22’ E."

So, do I understand correctly or am I backwards?

That’s exactly right yes.

[Capt Codetrucker]

That’s exactly right yes.

Thank you kindly. Now that I understand how this works, I should easily incorporate it in a SHtoStellarium script modification or a linked script. I'll post it once I get it working.

[don1reed]

Sorry, if I got to this late; but, is this what you're looking for? (It's a script written for PYTHON 34, by :

# By Michael Jones, 01/01/2008. mjones004@ameritech.net
# Revised 11/29/2009 to work with Stellarium 0.10.2
# This program reads the latitude and longitude data for a
# sub in Silent Hunter III, IV and creates a startup script for
# Stellarium. This sets the Date, Time and Location of the
# sub in Stellarium, allowing players to use celestial navigation in the game.

import string, os, sys

####### Edit Locations here ############

# Change the directory on the next line to point to SaveData.map in your saved game directory.
savedgame = "C:\\Users\\Donald\\My Documents\\SH4\\data\\cfg\\SaveGames\\00000000\\Sa veData.map"
# Set the Stellarium directory
stellocation = "C:\\Stellarium"

####### End Editable locations #########

# read savegame.
f=open(savedgame,'r')
content= f.read()
f.close()
# find lat & Lon in the file
start = content.find("Waypoint")
longst = content.find("Pt0=",start)
longnd = content.find(",",longst)
latnd = content.find(",",longnd+1)
# convert to dec deg
longitude = float(content[longst+4:longnd])/120000
latitude = float(content[longnd+1:latnd])/120000

# get date&time (input)
date = input('GMT yyyy/mm/dd :')
time = input('GMT hh:mm :')

# set the ocean label
if (-90 < longitude <= 30):
ocean = "Atlantic Ocean"
elif (30 < longitude <= 105):
ocean = "Indian Ocean"
elif (longitude <= -90 or longitude > 105):
ocean = "Pacific Ocean"

# Write startup script.
date = date.replace('/', ':')
longitude = str(longitude)
latitude = str(latitude)

stella=open(stellocation + "\\scripts\\sh4.ssc",'w')
stella.write("core.setDate(""+date+"T"+time+":00")\n")
stella.write("core.setObserverLocation("+longitude+", "+latitude+", 2, 0, "SH4 Navigation Point, "+ ocean +"", "earth")");
stella.close()

# now open stellarium and take your star sights
prog = "start /d ""+ stellocation +"" stellarium.exe --startup-script sh4.ssc"
os.system(prog)
sys.exit()

[don1reed]

IRL Before GPS, it was the Navigator's task to be on deck with the sextant for his call to prayer, five times per day:

Twilight sextant sights are best made between:

Nautical twilight: Sun 12° below horizon.
Civil Twilight: Sun 6° below horizon.

1) Morning twilight
2) Morning Sun sight
3) Noon Sun sight
4) Afternoon Sun sight
5) Evening twilight

As mentioned above, doing that in game will wear you out, especially if using math sight reduction methods, let alone the time consumption.
Cheers,

[Capt Codetrucker]

Hello,

In case anyone here is not monitoring Discord, the project to integrate Stellarium as a SH4 CellNav platform is continuing. Unfortunately, it has hit a snag. I am confident this effort can be successful if I can discover where to find the date and time of the last/saved position. The position coordinates of the script is successful, but as you know, coordinates without correct time is useless in CelNav.

Can someone point me to the exact file where SH4 saves the exact in-game date and time? I have been looking in every file I can see for two days.

Thanks.

[propbeanie]

I have looked for similar data amongst the files before, and my best guess is that it is in one of the binary files, or which I know nothing of the structure. If you look in the CareerTrack.upc file and notice the start location, they use "scientific" number notation, and not a straigt-up number, so I don't know if you would have to look for the long/lat stored the same way in a binary file, or the regular decimal location data. If I'm not mistaken some of those files are encrypted, which would make finding the data rather daunting. Also, a BIG problem with trying to use the Save data is having the player put an executable in the correct folder to gather the required data... It might be easier to have a little pop-up that the player inputs his location into prior to exiting the game... ?? Of course, one has to be careful shelling-out to Windows from the game, since that can cause trouble on some set-ups. :salute:

P.S.: it might be that Jeff-Groves could think of a way to "search" through the Save folder binary data with the 010 Editor program, and might make short work of the search...

[Capt Codetrucker]

@propbeanie
I have been wondering about the idea of a standalone app instead of reading SH. I might even make an iOS and Android app, but that presents other problems. Mainly, Stellarium is only available on desktop platforms.

[propbeanie]

There are some "alternatives" for the Android and iOS platforms for the pc-based Stellarium, though I have no idea as to their suitability to the task you're after. It would be nice to have a tablet or fone that could take user input of user data, and plot location... it would be even better if the game could allow for that, wouldn't it?... :arrgh!: - maybe someday. :salute:

https://www.androidfreeware.net/download-stellarium.html

[Capt Codetrucker]

Yes, maybe someday. Actually, I have been ruminating on how far this project might go. I can already see several phases, but for now I am focusing on making the first phase of manual time entry work flawlessly, at least on my SH4. “Phase 2” will be to automate the process which I should be able to do since I found the date/time data in the SaveGame.crp file. It is the log data which is produced that you see when a command is echoed back or a report is given. The file gets longer with every announcement, so the Phase 2 script will have to read to EOF and backread to get the data. It’s doable. The good news is the Phase 1 script may(?) get completed today. 😊

[Capt Codetrucker]

Hi,

I wanted to announce the script we have been working has gone gold! I started a new thread for it here…
https://www.subsim.com/radioroom/showpost.php?p=2764388&postcount=1

I want to offer a GREAT BIG THANK YOU to all you who lent a helping hand through this project. I could not have done it without you!

See you in the other thread. 😁👍