GR2 Editor/Viewer/Extractor/Importer

[Targor Avelany]

Originally Posted by TheDarkWraith

After this next version is completed I'll look into the reported Texture issues. The more data everyone can give me about whatever problems they are having with this the better. I need files, write ups of the problem, what you did to make the problem appear, etc. so that I can try and duplicate it and single step it in Visual Studio's debugger to try and figure out why

That - will not be a problem :-)
I'll run through various options over this weekend and will write up all the details step by step.

[gap]

Originally Posted by Targor Avelany

On the gap's post: yeah, it is possible that a lot of them will have to be re-done. I also wonder if the above is the cause for my texture problems.

Originally Posted by TheDarkWraith

Depends. If there was any extendeddata that had a granny_struct_pointer that had a null value then yes. There is an easy way to tell if your file is broken and/or corrupted. Load the file in question with the app. After it's loaded and displayed press F1 to bring up the Sentinel window...

Thanks to both of you for the explainations

Originally Posted by TheDarkWraith

After this next version is completed I'll look into the reported Texture issues. The more data everyone can give me about whatever problems they are having with this the better. I need files, write ups of the problem, what you did to make the problem appear, etc. so that I can try and duplicate it and single step it in Visual Studio's debugger to try and figure out why

Probably unrelated, but have you seen my report in post #1753?

[TheDarkWraith]

Fixed more problems found in previous versions.

Wanted to record information about the decompression in the GR2 file and found the perfect place to put it: Under the ExtendedData of the main menu Since I've never seen this populated in any GR2 file I've opened I decided to put it to good use (and even if the app finds it populated it will just add more entries to it)

http://www.mediafire.com/view/?e3t1cqbh533m25g

Once you click the ExtendedData (and if the file is decompressed) you'll see:

http://www.mediafire.com/view/?0rk1wjlr7fl9ha8

Clicking the Decompression link will show you the decompression information:

http://www.mediafire.com/view/?81qbckwicqc08sa

I'm finally coding in my Author Info idea. This will let the author of the file record information about it in the GR2 file. This will be added to the main menu's ExtendedData also but the link will say 'Author Info'. In this section I plan on letting the author add name, contact info, notes, description, whether the file is viewable by the app, whether the file is editable by the app, whether the file is viewable by the Granny Viewer or not. Since my application is the only one that can currently edit and view GR2 files outside of Rad Games Tools apps this will give the author the options of how tight they want their work protected. If the author chooses to not let the file be editable by the GR2 Editor/Viewer then they will need to supply a password to protect it. The app will check to see if the file is marked as editable or not and if it's not the end user will only be able to view the file with this app (if the author chooses to let them view it). Viewing the file is just as it means - you can only view it (you cannot extract, change, edit, zip, nada, nothing). If the author chooses to not let you view the file then the app will enforce this and not let you load the file in question (unless you supply the correct password).

I plan on letting the author specify two passwords: one for viewing and one for editing. This way the author can give the viewing password to others so they can view their work but not make any changes to it.

Originally Posted by TheDarkWraith

Fixed more problems found in previous versions.

Wanted to record information about the decompression in the GR2 file and found the perfect place to put it: Under the ExtendedData of the main menu Since I've never seen this populated in any GR2 file I've opened I decided to put it to good use (and even if the app finds it populated it will just add more entries to it)

http://www.mediafire.com/view/?e3t1cqbh533m25g

Once you click the ExtendedData (and if the file is decompressed) you'll see:

http://www.mediafire.com/view/?0rk1wjlr7fl9ha8

Clicking the Decompression link will show you the decompression information:

http://www.mediafire.com/view/?81qbckwicqc08sa

I'm finally coding in my Author Info idea. This will let the author of the file record information about it in the GR2 file. This will be added to the main menu's ExtendedData also but the link will say 'Author Info'. In this section I plan on letting the author add name, contact info, notes, description, whether the file is viewable by the app, whether the file is editable by the app, whether the file is viewable by the Granny Viewer or not. Since my application is the only one that can currently edit and view GR2 files outside of Rad Games Tools apps this will give the author the options of how tight they want their work protected. If the author chooses to not let the file be editable by the GR2 Editor/Viewer then they will need to supply a password to protect it. The app will check to see if the file is marked as editable or not and if it's not the end user will only be able to view the file with this app (if the author chooses to let them view it). Viewing the file is just as it means - you can only view it (you cannot extract, change, edit, zip, nada, nothing). If the author chooses to not let you view the file then the app will enforce this and not let you load the file in question (unless you supply the correct password).

I plan on letting the author specify two passwords: one for viewing and one for editing. This way the author can give the viewing password to others so they can view their work but not make any changes to it.

Holy clown face batman!! When did you get back TDW!?!?

[Targor Avelany]

Nice!!!

[TheDarkWraith]

when the app spawns the Granny Viewer it will change the title of the Granny Viewer app to reflect it's current purpose. It will also have the Granny Viewer app spawn a MessageBox stating what to do:

http://www.mediafire.com/view/?1zeboyjlz76ptbb



EDIT:

I'm more than likely going to change this. Since I know where in the app it goes to read the string of the filepath you selected I will write the necessary string value to that memory address and just bypass the openfiledailog box. That way you just have to press Load file... and sit back and watch the show...

[Ekmek]

great work!

[Ekmek]

Originally Posted by TheDarkWraith

when the app spawns the Granny Viewer it will change the title of the Granny Viewer app to reflect it's current purpose. It will also have the Granny Viewer app spawn a MessageBox stating what to do:

http://www.mediafire.com/view/?1zeboyjlz76ptbb



EDIT:

I'm more than likely going to change this. Since I know where in the app it goes to read the string of the filepath you selected I will write the necessary string value to that memory address and just bypass the openfiledailog box. That way you just have to press Load file... and sit back and watch the show...

So the granny viewer will be the "muscle" for loading gr2s now? Basically anything you can open in granny viewer can be modified?

I ask because I found an issue with my custom gr2s were the civ5 ones have something called "submaps" in their materials but i cant duplicate them. I think this is what you mean by embedding the textures.

[flighttools]

The granny viewer can take a command line argument of the file to load. Have you tried that instead of playing with memory addresses?

[TheDarkWraith]

Originally Posted by Ekmek

So the granny viewer will be the "muscle" for loading gr2s now? Basically anything you can open in granny viewer can be modified?

I ask because I found an issue with my custom gr2s were the civ5 ones have something called "submaps" in their materials but i cant duplicate them. I think this is what you mean by embedding the textures.

The Granny Viewer is only spawned and used when the app needs to decompress the GR2 file the user asked it to load. The app still does all the heaving lifting of reading/parsing/creating the GR2 file.

I have changed how the Granny Viewer works now also. Now a dialog box will show saying you only have to click load file... to begin. The granny viewer app will close itself down automatically when it has finished decompressing the GR2 file. Why do you have to press Load file... at all? To avoid legal issues/problems. I could easily cut the code out of Granny Viewer that does the decompressing and put it into a DLL or even the app but doing so could/would raise legal problems. By using the Granny Viewer app and only modifying process memory (and not the exe itself) I avoid all legal problems. By having to click Load file... you are still using the app as it was intended (only slightly modified for what I want it to do).

Your Granny Viewer app on your computer will remain unharmed. All the changes I do to the Granny Viewer are done in process memory after the winloader has placed it in your computer's memory (basically after windows has finished creating the process and is 'running' the app). This means all changes to the Granny Viewer app are lost when it closes itself. And before anyone asks, no you can't stop the Granny Viewer from closing itself down. The code I inject/change forces it to terminate after it decompresses all the needed sections.

These submaps you are referring to are the hidden textures and materials in the GR2 file. The app will expose them so you can edit/view them (though they will remain hidden when the app writes the file - this means not viewable in Granny Viewer) I might code something in in later versions to make these invisible textures and materials visible. It's just a simple matter of adding a pointer for each invisible one to expose it.

Originally Posted by flighttools

The granny viewer can take a command line argument of the file to load. Have you tried that instead of playing with memory addresses?

Yes I know it will take command line arguments. When I was REing the Granny Viewer app one of the first things it does is read the command line arguments.

I had to write the path to the Granny Viewer memory because I didn't want the user to be able to pick the wrong file. Now when you click load file... the granny viewer automatically starts loading the file I wrote into it's memory. Thus no user error can occur

You have to think about how the end-user can screw up and code for it (or try to at least)

Version 1.1.314.1 is complete and is undergoing final testing. The author info idea will not be in this version. This version will be able to decompress compressed GR2 files and it contains some bug fixes for the bugs I was able to find. Expect to see a release date of tomorrow (Saturday) for it if all goes well

I was able to load the Otto_Bismark_OLD.GR2 file also. The Decompressor is working perfectly so far

The app's name has changed to GR2 Editor/Viewer as it is no longer confined to just SH5 GR2 files.

The app is unable to decompress a GR2 file that is compressed AND has pointers before data. The reason being is I haven't been able to test it with a GR2 file like this. I need a GR2 file like this so I can verify the decompressor works as it should with this special type of GR2 file. If you find one (the app will tell you if you try to decompress one) please send it to me.

In the next version I hope to have some code/window in place so that you can specify to the app which mapusage is for diffuse, bump, ambient-occlusion, etc. Currently I have them hard coded to SH5 values.

Your anti-virus software may complain about the app now. Reason being is the code I have for modifying process memory. I use this same code to change/extract data/code from apps loaded on other's computers. It's very good at what it does! The best part is the app it's changing/reading/modifying has no idea that anything ever happened to it

[TheDarkWraith]

Might be a slight delay in releasing new version. Followed up on a note I made to myself about some data that I found that I didn't really know it's purpose. I found this data in the Augustus.GR2 file and also in the Undine.GR2 file. After playing with a bunch of numbers trying to get something to work out I finally figured out that this data is a special type of datatype definition. It's exact purpose is still unclear but they are definitely a type of datatype definition. What does this mean? This means that I haven't been updating them This really isn't a problem unless a section in your GR2 file has this (referring to unknown_count):

[8528] Reading section 1 (vertex_data) data starting at 0x94
[8528] Compression Type=0x0 0x94 always 0x0
[8528] Section file offset=0x4D534 0x98 where this section starts in the file
[8528] Compressed size=60992 0x9C compressed size and uncompressed sizes should equal for SH5 GR2s
[8528] Uncompressed size=60992 0xA0 compressed size and uncompressed sizes should equal for SH5 GR2s
[8528] Alignment=0x4 0xA4 what the alignment boundary is in the file
[8528] Start of data offset=0x780 (0x4DCB4) 0xA8 start of data
[8528] Start of data offset2=0x780 (0x4DCB4) 0xAC start of data
[8528] Pointers file offset=0x1189AC 0xB0 where the pointers for this section are located
[8528] Number of pointers=0 (0x0) 0xB4
[8528] End of pointers file offset=0x11C648 0xB8 where the pointers for the file end
[8528] Unknown_Count=0x1 0xBC
[8528] Current position in file is 0xC0

unknown_count used to be an unknown to me but I finally figured out that it is the number of these special datatype definitions in the file. If you total all these unknown_counts for all the sections and multiply by the size of this data structure it works out to be what it should be I've figured out the data structure for the most part, only 0x4 bytes of it I'm still scratching my head over. There are two 0x4 byte parts of this data structure that I have to update when Sentinel makes any changes to the file. I'll have to add code to my pointer updater code to handle these special data structures.

Now the purpose of these special datatype definitions is still unclear to me. I've found where they show in the Granny Viewer but it still doesn't make any sense to me yet

[TheDarkWraith]

Figured these special datatype definitions out They are describing vertex data, specifically vertex data that has mixed values (Real32 and UInt8 mixed together for instance). All the vertex data that had the same values (Real32 for instance) didn't 'trigger' any of these special datatype definitions. These also play a part in the section's data offset. Wow. More error checking and code needed in my app now These things are very important and I've been ignoring them

[Ekmek]

Thats great. I cant wait to test this out!

[gap]

Looking forward to your next update TDW

When you get time, can you please have a look at this post?

http://www.subsim.com/radioroom/show...&postcount=106

[TheDarkWraith]

Originally Posted by gap

Looking forward to your next update TDW

When you get time, can you please have a look at this post?

http://www.subsim.com/radioroom/show...&postcount=106

Uncheck (or check) use local settings in the options (Edit-->Options). If that doesn't work post a link to the files here so I can see what's going on in Visual Studio's debugger.


These new special datatype definitions finally unlocked what sections 3 and sections 4 are in the GR2 file: deformable vertex data and deformable indices respectively. That means sections 1 and 2 are rigid vertex data and rigid vertices respectively Another piece to the puzzle added........but the mystery deepens Adding all the new error checking code in for these special datatype definitions revealed that all the vertex data I'm currently reading from Augustus.GR2 is deformable. I'm reading no rigid data This is not the apps fault, it's because this data is being hidden in the GR2 file. Now just have to figure out how to find it...