[TEC] SH5.exe patches to fix bugs and add functionality

[gap]

Originally Posted by TheDarkWraith

Gives you some insight into the NT structure and PE format in general

v1.0.29.0 released. See post #1

Major overhaul to the patcher app. One, it's now generic so can be used to patch any file. Two, it's much more user friendly. Three, you can add you own sections now (and the Windows loader will load them in)

Hi TDW,

long time ago I was a member of the GPx modding community. I remember one of its most respected members was a certain SDI, in the world René Smit. He was the author of GPxPatch: a real time extension of GP3/GP4 fixing many bugs and adding several new features to the game.

In my opinion its best feature was not to change any executable: it was changing the code of the game after it was loaded in memory, allowing full configuration by the player. Furthermore, the patch was enabling third parties custom extensions to be loaded and executed within the game. For what I can understand, he was "injecting" his code in the game as you did with dll's with your dll injector.

Knowing your love for neat solutions, and seeing the you are moving in the same direction, I wonder if the same can be done with a future release of your patcher.

[TheDarkWraith]

Originally Posted by gap

In my opinion its best feature was not to change any executable: it was changing the code of the game after it was loaded in memory, allowing full configuration by the player. Furthermore, the patch was enabling third parties custom extensions to be loaded and executed within the game. For what I can understand, he was "injecting" his code in the game as you did with dll's with your dll injector.

Knowing your love for neat solutions, and seeing the you are moving in the same direction, I wonder if the same can be done with a future release of your patcher.

You want the patcher to be able to inject code into an active process? I wrote a piece of software than can do that already. But you want that functionality included in the patcher?

Let me ask you this: why inject code into an active process when you can take it's .exe or .dll file, add a section or multiple new sections to it, dump your code into the new section (or sections) in the file, and let the Windows loader do the loading for you? With the adding new sections your code is guaranteed to be where you want it thus making it very easy to get to the other sections. With code injection you have Windows create a new pool of memory for you and you dump your code into that pool. Where that pool of memory ends up being located you have no control over. The only reason I can see for not adding new sections and dumping your code into them in the .exe or .dll file is if the .exe or .dll checks itself when loaded to see if anything is different. But then again one can remove this check from ever happening.

If the process is already active (loaded) and you don't have access to the .exe or .dll files then code injection is necessary if one wants to add anything new to it.

Looking at GPxPatch I wouldn't like that you have to launch the app wanting to patch from the patch app itself. There's no reason to do this. You can easily get a list of all active processes from Windows and 'attach' to any of them very easily.

The patcher has been morphing into something I've been wanting to do for a long time: take an active process and grab all of it's in memory data. Since the PE header is in the process's memory one can theoretically read this PE header and construct an .exe or .dll from the memory read in (a reverse Windows loader). For self-modifying exes this would be one way to 'capture' the image of it in it's modified state thus removing the obfuscation. That's the theory anyway. I'm slowly making my way to see if this theory is correct or not The only real hole in this theory is if a section was marked as discardable and the Windows loader didn't load that section into memory...

I can dump memory from any active process, modify it, and inject it back in and the process has no idea anything happened. The patcher is heading in that direction as another way to patch (basically taking functionality from another app I wrote).

I've been really ambitious lately and have been trying to modify the sections already defined in an .exe or .dll file. The new version of the patcher allows you to add new sections. These sections are added to the end of the .exe or .dll file. That's cool and all but I want to be able to extend sections already defined The problem with extending sections already defined is relocations, fix-ups, and the likes. While not an impossible task to do it's not for the faint of heart. I'm learning about relocations, fix-ups, and all the other things I need to know (data structures) to be able to pull this off. I've searched for tools that already do this and haven't found any. When I've posed this question in some forums the response back was impossible or you need the source code...well I always love to prove that nothing is impossible

Another thing I've been working on is being able to 'add' a process to another process. You may ask why would one want/need something like this? Well I have one reason: the Enigma app. The app is perfect replica of an Enigma machine. It works great as a stand-alone process. It cannot be used in any app that does full screen rendering. I've written an app that can 'steal' the main window away from another process and modify it so that it thinks it's the child of the 'stealing' process. Works beautifully. You minimize the 'stealing' process and the other process minimizes also. Now I just have to modify it (Enigma) so that it renders it's window over the full screen app. This is where I need to be able to attach the Enigma process to the full screen process.

[gap]

Originally Posted by TheDarkWraith

You want the patcher to be able to inject code into an active process? I wrote a piece of software than can do that already. But you want that functionality included in the patcher?

Let me ask you this: why inject code into an active process when you can take it's .exe or .dll file, add a section or multiple new sections to it, dump your code into the new section (or sections) in the file, and let the Windows loader do the loading for you?

If the process is already active (loaded) and you don't have access to the .exe or .dll files then code injection is necessary if one wants to add anything new to it.

Looking at GPxPatch I wouldn't like that you have to launch the app wanting to patch from the patch app itself. There's no reason to do this. You can easily get a list of all active processes from Windows and 'attach' to any of them very easily.

The patcher has been morphing into something I've been wanting to do for a long time: take an active process and grab all of it's in memory data. Since the PE header is in the process's memory one can theoretically read this PE header and construct an .exe or .dll from the memory read in (a reverse Windows loader). For self-modifying exes this would be one way to 'capture' the image of it in it's modified state thus removing the obfuscation. That's the theory anyway. I'm slowly making my way to see if this theory is correct or not

I can dump memory from any active process, modify it, and inject it back in and the process has no idea anything happened. The patcher is heading in that direction as another way to patch.

I've been really ambitious lately and have been trying to modify the sections already defined in an .exe or .dll file. The new version of the patcher allows you to add new sections. These sections are added to the end of the .exe or .dll file. That's cool and all but I want to be able to extend sections already defined The problem with extending sections already defined is relocations, fix-ups, and the likes. While not an impossible task to do it's not for the faint of heart. I'm learning about relocations, fix-ups, and all the other things I need to know to be able to pull this off. I've searched for tools that already do this and haven't found any. When I've posed this question in some forums the response back was impossible or you need the source code...well I always love to prove that nothing is impossible

it sounds very challengig!

I was asking if it was possible to inject code in the active process rather than modifying its files for two main reasons:
- out of curiosity;
- because I think it would be a cleaner approach than modifing the executable: the lesser common users deal with it, the lesser they risk to screw up things;

Anyway, if code injection demonstrated to be a valid approach, my next question was going to be:

- would your patcher allow us to change the injected code on the fly, while the game is running, i.e. enabling/disabling patches, changing some fixed parameters, etc?

[TheDarkWraith]

Originally Posted by gap

Anyway, if code injection demonstrated to be a valid approach, my next question was going to be:

- would your patcher allow us to change the injected code on the fly, while the game is running, i.e. enabling/disabling patches, changing some fixed parameters, etc?

Code injection is a very valid approach. It's used by MANY people to do whatever it is they want to achieve/discover.

The patcher, in it's current state, won't allow you to do what you are wanting (on the fly). I can add that functionality though as it's just merely taking code I've already written and adding it to the patcher app. What I do need to add to the patcher app is a way to dump memory from a process and reload that memory back in. This would allow you to say dump the initialized data from SHSim, modify it, load it back in and see what effect your changes have. Alright I just convinced myself to do this The more people playing around and discovering things the faster we map out where all the variables are located and what they do!

[gap]

Originally Posted by TheDarkWraith

Code injection is a very valid approach. It's used by MANY people to do whatever it is they want to achieve/discover.

The patcher, in it's current state, won't allow you to do what you are wanting (on the fly). I can add that functionality though as it's just merely taking code I've already written and adding it to the patcher app. What I do need to add to the patcher app is a way to dump memory from a process and reload that memory back in. This would allow you to say dump the initialized data from SHSim, modify it, load it back in and see what effect your changes have. Alright I just convinced myself to do this The more people playing around and discovering things the faster we map out where all the variables are located and what they do!

Thank you!

For sure adding such a functionality would boost significantly our knowledge of SH5 and indeed our modding potential: just think of how many parameters we could experiment with, without having to exit to windows, apply our changes, load again SH5, start a new game... cycle this process for n times, and you will just get sick

[gap]

Originally Posted by flostt

thanks for sharing this gap, very interesting to read, especially the failure rates at the beginning of the war and all the reasons for the failures

yep, in my opinion that's a very good reference. I am glad that you like it too!

[flostt]

When using TDW's Generic Patcher remember his note at the bottom of post #1 :

NOTE: DISABLE ALL PATCHES WITH OLD PATCHER BEFORE APPLYING NEW PATCHES WITH NEW PATCHER! THIS APPLIES ONLY TO PATCHES RELATED TO SHSIM.ACT AND SHCOLLISIONS.ACT

I read it and forgot about it......and it took me quite a while to get everything fixed again.....

[Deepsheep]

I have a steam version of the game. I'm sad the current patch will not work with my version, but it sounds like applying the fix at runtime would solve this issue as well.

[TheDarkWraith]

v1.0.31.0 released. See post #1

This version adds the ability to attach to a process and inject/remove DLLs.

To attach to a process selected File-->Attach. To detach from the process select File-->Detach.

To inject a DLL/ACT file click Inject button. To remove a DLL/ACT file from the process select the file you want to remove from the Loaded modules listview. Then click Remove.

Everything else should be pretty self-explanatory.

If it's unable to inject/remove it's probably because you have older version of kernel32.dll on your computer. Currently I have the offsets for LoadLibraryA and FreeLibrary hard-coded. I have to add code that will read the IAT to get it's address yet.

This can be run on 32bit or 64bit OSs. I have it coded to select the appropriate function calls based on whether your OS is 32 bit or 64 bit.

Working on the ability to inject/remove files that are nothing but hex code. Since I can't call LoadLibraryA and FreeLibrary on them (they're not DLLs) I have to code in something to allocate process memory and then dump the contents of those files into the new process memory.

[TheDarkWraith]

v1.0.33.0 released. See post #1

This version allows you to inject a file (ANY type of file) into a process. You also have the option of executing the file you loaded You can inject ANY file into a process (bmp, hex, txt, etc, dll, exe).

If you elect to execute the file you injected, the last command HAS to be a RETN statement (or a RETN x if you need to pop temp variables off the stack). If you forget this the process WILL CRASH! You don't have to add the RETN statement if you are going to branch to somewhere else in the process but in doing so don't forget to pop the return value pushed onto the stack by the stub!

The execution will begin at the first byte found in the file. I'll add a way for you to specify beginning execution point in next version.

I've included a little file (TestFile(retn).hex) that you can inject into the SH5 process. When it asks if you want to execute it say yes. SH5 will never know anything every happened! The file is just one command, RETN, which means return to caller. Here's how the inject file works:
- the app allocates memory in the process in a multiple of pagesize to fit the file being injected
- the file contents are copied into this memory
- if the user selected execute then:
- a stub is made that will branch execution to the injected file. The first thing the stub does is push the EIP to the stack. This way when the injected code issues a RETN statement execution goes back to it's original place

There are two new buttons - Inject file and Remove file. They do just as they sound. To remove a file select the file from the Injections listview and then press Remove file.

Working on dumping process memory and reloading it now so user can modify it. Also adding a way to set watches on memory and letting you define the update time interval on them

The LoadLibraryA and FreeLibray offsets are still hardcoded for kernel32.dll. If you get failed to inject or failed to remove it's probably because you have older version of kernel32.dll installed on your computer. I should have these hard codes removed in next version.

[quink99]

Originally Posted by TheDarkWraith

v1.0.33.0 released. See post #1

This version allows you to inject a file (ANY type of file) into a process. You also have the option of executing the file you loaded You can inject ANY file into a process (bmp, hex, txt, etc, dll, exe).

If you elect to execute the file you injected, the last command HAS to be a RETN statement (or a RETN x if you need to pop temp variables off the stack). If you forget this the process WILL CRASH! You don't have to add the RETN statement if you are going to branch to somewhere else in the process but in doing so don't forget to pop the return value pushed onto the stack by the stub!

The execution will begin at the first byte found in the file. I'll add a way for you to specify beginning execution point in next version.

I've included a little file (TestFile(retn).hex) that you can inject into the SH5 process. When it asks if you want to execute it say yes. SH5 will never know anything every happened! The file is just one command, RETN, which means return to caller. Here's how the inject file works:
- the app allocates memory in the process in a multiple of pagesize to fit the file being injected
- the file contents are copied into this memory
- if the user selected execute then:
- a stub is made that will branch execution to the injected file. The first thing the stub does is push the EIP to the stack. This way when the injected code issues a RETN statement execution goes back to it's original place

There are two new buttons - Inject file and Remove file. They do just as they sound. To remove a file select the file from the Injections listview and then press Remove file.

Working on dumping process memory and reloading it now so user can modify it. Also adding a way to set watches on memory and letting you define the update time interval on them

The LoadLibraryA and FreeLibray offsets are still hardcoded for kernel32.dll. If you get failed to inject or failed to remove it's probably because you have older version of kernel32.dll installed on your computer. I should have these hard codes removed in next version.

TDW, I was wondering if you might have a Child's Garden-type explanation of your revised MOD above of what this revision is and what it does spoken slowly and using small words for those computer poster children among'st us.

Thanks

[BIGREG]

To see this is great , but it is beyond my knowledge

What this mean in concrete

The possibility of adding what ?

- Contollers for. SIM? (Dials, lights ....)

- Otherwise to open other programes eg: Enigma, sextant ...

- .....

[volodya61]

Originally Posted by quink99

TDW, I was wondering if you might have a Child's Garden-type explanation of your revised MOD above of what this revision is and what it does spoken slowly and using small words for those computer poster children among'st us.

Thanks

Absolutely agree
I would say the same thing, but couldn't find the right words.. my poor English..
I don't understand it myself (my poor English), and I can't explain this in sukhoi.ru..
TDW! Maybe you can make a simple version for children, please!

[Endroo]

Oh my! I can't seem to find the older patches, but the CO2 fix in the latest asks for it. What do I do?

[TheDarkWraith]

Originally Posted by BIGREG

To see this is great , but it is beyond my knowledge

What this mean in concrete

The possibility of adding what ?

- Contollers for. SIM? (Dials, lights ....)

- Otherwise to open other programes eg: Enigma, sextant ...

- .....

Originally Posted by volodya61

Absolutely agree
I would say the same thing, but couldn't find the right words.. my poor English..
I don't understand it myself (my poor English), and I can't explain this in sukhoi.ru..
TDW! Maybe you can make a simple version for children, please!

In laymen's terms I'm giving you the tools to add/expand/change a process. If you're not familiar with what a process is it's a Windows term - once again, in laymen's terms, think of it as the encapsulation of everything needed to run an application in windows.

DLL injection let's you write code in some editor (Visual Studio for instance), compile that code into a DLL, and then inject that into a process to either change it's functionality, override a functionality, and/or add new functionality. Now simply injecting a DLL isn't going to do much. Yes, the DLL is now part of the process but it's never being 'called'. You need to add hooks into the existing code so that your new code is called.

If you're really computer saavy you can replace an existing DLL with your own. That lets you do things the app never intended to do (DLL injection/code injection does also but this way is easier for some)

Injecting a file is just as it sounds - taking the contents of a file and placing it inside an active process. What does this do for you? I can think of many things! You all are familiar with my patches - you load up the patch file and then you enable the patches you want. Think of those patches as files being injected into the SH5 process (I actually write new data to existing files but it should help you understand the concept).

Now for those of you, like me, who love assembly (long live DOS!) injecting a file is the way to go. You can write your 'code' in an assembler, take the hex file generated and inject it into the process. Once I code in the ability to dump and replace process memory then you will have an easy way to add your hooks to call your new code.

I'm working on the ability to add watches currently. If your familiar with Visual Studio then you know what watches are already. If not I'll try to make it a little clear: say you have an application that displays a value on a screen and you'd really like to change that value and/or monitor it. We have no idea where this value resides so we have to do a little exploring inside the active process. We pick an area of memory and set a watch on it - the watch will monitor that area of memory for any changes. When it detects changes it can either suspend the process or just update the app with the new value. This allows you to see what that memory is doing in real time. By setting many different watches (and changing what they are monitoring) you will eventually find where the variable is located so you can manipulate it.

I made the app generic in nature so that it can be used on anything. It's not fixed to any particular app or process.

This app now let's you 'play' in my favorite playground...process memory. You are limited only by your imagination/skill level