Crash: comments at the start of ini files considered harmful.

a.g. · 11-11-15, 02:50 PM

I had my first consistent game crash in real play within a certain area, and investigated. I turns out, the code that is used by the game to read ini-style files has a bug: if there are comments before the first [...] label, it can rarely and semi-randomly cause one such section within the file to be effectively emptied of all contents. If it happens to an .eqp file, the game will crash whenever the affected ship tries to be rendered, whether in mission or in Museum (in my case, the damage was to Sea\NTRW_\NTRW_.eqp for GWX's ASW Trawler).

This bug can be fixed with a relatively small patch, but an easier workaround may be to never use comments at the very start of an ini file.

The technical cause of the bug is as follows: the ini reader uses hashes to index ini sections, without any support for collisions; it also for some reason reads and stores comments into memory. If a comment occurs at the start of the file, it creates an empty section without a name to store it, and does not initialize the hash value for that empty section. If the memory happens to contain a value that matches the hash of one of the real section names, the collision will prevent any of the values within it to be read.

This is the patch that worked for me (it's for the Steam version in case it matters):

GT182 · 11-11-15, 08:31 PM

There's could be the problem... Steam. I have a few games on Steam that I had before on my computer from the DVDs. No affiliation with Steam at all back then. Mods for these won't work with a Steam install. Looks to be the same for SH3 running thru Steam.

One game was X3 Terran Conflict. Mods for the DVD install will not work for the Steam install. And that's one of the reason I do not like Steam. Mods must be made to be compatable with Steam.

GT182 · 11-11-15, 08:34 PM

That's the problem... Steam. I have a few games on Steam that I had played before on my computer from the DVDs. This is before I knew Steam even existed. No affiliation with Steam at all back then. Mods for these won't work with a Steam install. Looks to be the same for SH3 running thru Steam.

One game was X3 Terran Conflict. Mods for the DVD install will not work for the Steam install. And that's one of the reason I do not like Steam. Mods must be made to be compatable with Steam. Like done with X Rebirth, which can only be played thru Steam.

a.g. · 11-12-15, 03:05 AM

The only reason I mentioned Steam is that if for some reason it has a different version of that dll, the patch won't work in non-steam versions. However, the dll has a build timestamp in 2005, so that is unlikely.

LGN1 · 11-13-15, 04:55 PM

Hi a.g.,

are empty lines or lines with just a space also a problem? And is there a way to test your findings in one's own installation?

I have just checked. GWX has many such files with comments at the top

Regards, LGN1

a.g. · 11-14-15, 03:27 AM

If I understand the disassembly correctly, only actual comments located before the first [...] should matter, whitespace is fine.

As an uninitialized variable bug, this belongs to a category that is notoriously difficult to reproduce intentionally: there are so many things that can affect what was stored in that memory location previously, from the list of mods, to details of implementation of malloc in system libraries. On my computer without the patch I just need to open Museum and click on New Zealand -> ASW Trawler, and it crashes in msvcrt71.dll with a null pointer access.

Apart from being able to reproduce the remote effects, there is the fact that that particular function in FileManager.dll has a code path where it doesn't initialize a memory location, but still reads it later - that is a definite bug

I also made a patch kit for applying or removing the patch more conveniently:

https://www.dropbox.com/s/jwjgutc0cs...Crash.zip?dl=1

11-11-15, 02:50 PM	#1
a.g. Swabbie Join Date: Nov 2015 Posts: 11 Downloads: 18 Uploads: 0	Crash: comments at the start of ini files considered harmful. I had my first consistent game crash in real play within a certain area, and investigated. I turns out, the code that is used by the game to read ini-style files has a bug: if there are comments before the first [...] label, it can rarely and semi-randomly cause one such section within the file to be effectively emptied of all contents. If it happens to an .eqp file, the game will crash whenever the affected ship tries to be rendered, whether in mission or in Museum (in my case, the damage was to Sea\NTRW_\NTRW_.eqp for GWX's ASW Trawler). This bug can be fixed with a relatively small patch, but an easier workaround may be to never use comments at the very start of an ini file. The technical cause of the bug is as follows: the ini reader uses hashes to index ini sections, without any support for collisions; it also for some reason reads and stores comments into memory. If a comment occurs at the start of the file, it creates an empty section without a name to store it, and does not initialize the hash value for that empty section. If the memory happens to contain a value that matches the hash of one of the real section names, the collision will prevent any of the values within it to be read. This is the patch that worked for me (it's for the Steam version in case it matters): Code: Zero out the whole structure, instead of just 4 fields: .text:10005C72 cmp ebx, 0FFFFFFFFh .text:10005C75 mov [esi+0Ch], edi .text:10005C78 mov [esi+10h], edi .text:10005C7B mov [esi], edi .text:10005C7D mov [esi+4], edi ==> .text:10005C72 mov edi, esi .text:10005C74 mov ecx, 5 .text:10005C79 xor eax, eax .text:10005C7B rep stosd .text:10005C7D cmp ebx, 0FFFFFFFFh ------ This difference file is created by The Interactive Disassembler FileManager.dll 00005C72: 83 89 00005C73: FB F7 00005C74: FF B9 00005C75: 89 05 00005C76: 7E 00 00005C77: 0C 00 00005C78: 89 00 00005C79: 7E 31 00005C7A: 10 C0 00005C7B: 89 F3 00005C7C: 3E AB 00005C7D: 89 83 00005C7E: 7E FB 00005C7F: 04 FF

11-11-15, 08:31 PM	#2
GT182 Ocean Warrior Join Date: May 2005 Location: New Castle of Delaware Posts: 3,231 Downloads: 658 Uploads: 0	There's could be the problem... Steam. I have a few games on Steam that I had before on my computer from the DVDs. No affiliation with Steam at all back then. Mods for these won't work with a Steam install. Looks to be the same for SH3 running thru Steam. One game was X3 Terran Conflict. Mods for the DVD install will not work for the Steam install. And that's one of the reason I do not like Steam. Mods must be made to be compatable with Steam. __________________ Gary No Borders, No Language, No Culture =s No Country I'm a Deplorable, and proud of it.

11-11-15, 08:34 PM	#3
GT182 Ocean Warrior Join Date: May 2005 Location: New Castle of Delaware Posts: 3,231 Downloads: 658 Uploads: 0	That's the problem... Steam. I have a few games on Steam that I had played before on my computer from the DVDs. This is before I knew Steam even existed. No affiliation with Steam at all back then. Mods for these won't work with a Steam install. Looks to be the same for SH3 running thru Steam. One game was X3 Terran Conflict. Mods for the DVD install will not work for the Steam install. And that's one of the reason I do not like Steam. Mods must be made to be compatable with Steam. Like done with X Rebirth, which can only be played thru Steam. __________________ Gary No Borders, No Language, No Culture =s No Country I'm a Deplorable, and proud of it.

11-12-15, 03:05 AM	#4
a.g. Swabbie Join Date: Nov 2015 Posts: 11 Downloads: 18 Uploads: 0	The only reason I mentioned Steam is that if for some reason it has a different version of that dll, the patch won't work in non-steam versions. However, the dll has a build timestamp in 2005, so that is unlikely.

11-13-15, 04:55 PM	#5
LGN1 Ace of the Deep Join Date: Mar 2006 Posts: 1,138 Downloads: 147 Uploads: 12	Hi a.g., are empty lines or lines with just a space also a problem? And is there a way to test your findings in one's own installation? I have just checked. GWX has many such files with comments at the top Regards, LGN1

11-14-15, 03:27 AM	#6
a.g. Swabbie Join Date: Nov 2015 Posts: 11 Downloads: 18 Uploads: 0	If I understand the disassembly correctly, only actual comments located before the first [...] should matter, whitespace is fine. As an uninitialized variable bug, this belongs to a category that is notoriously difficult to reproduce intentionally: there are so many things that can affect what was stored in that memory location previously, from the list of mods, to details of implementation of malloc in system libraries. On my computer without the patch I just need to open Museum and click on New Zealand -> ASW Trawler, and it crashes in msvcrt71.dll with a null pointer access. Apart from being able to reproduce the remote effects, there is the fact that that particular function in FileManager.dll has a code path where it doesn't initialize a memory location, but still reads it later - that is a definite bug I also made a patch kit for applying or removing the patch more conveniently: https://www.dropbox.com/s/jwjgutc0cs...Crash.zip?dl=1