GR2 Editor/Viewer/Extractor/Importer

[TheDarkWraith]

Originally Posted by Vanilla

Then I guess we will be able to add waypoints for the crew without using a hex-editor as well! That's just incredible!

I'm writing the allocater/deallocater currently. These two functions in my sentinel class will allocate or deallocate space in the GR2 file. This will allow one to add to and/or remove items from a GR2 file. Should be writing my first GR2 file with new information in it shortly. It has to pass the Granny Viewer test. If it passes then everything is working as it's supposed to

[TheDarkWraith]

The allocater successfully allocated 500 bytes in a GR2 file. One major step forward. There was a slight problem with remapping the pointers. I forgot a major pointer in a section (forgot to read it in when reading the file) and that hosed everything else up in that section (didn't pass Granny Viewer test). I just have to account for this major pointer and everything should be golden.

[longam]

Originally Posted by TheDarkWraith

The allocater successfully allocated 500 bytes in a GR2 file. One major step forward. There was a slight problem with remapping the pointers. I forgot a major pointer in a section (forgot to read it in when reading the file) and that hosed everything else up in that section (didn't pass Granny Viewer test). I just have to account for this major pointer and everything should be golden.

Silent hunter 6 on its way! No wait, super mod I mean.

[TheDarkWraith]

Today v1.0.530.0 of the GR2Editor/Viewer graduated Kindergarten...it's growing up so fast! It wrote it's first GR2 file that passed the granny viewer test and has no anomolies or defects that I can find

I took the lifeboat GR2 file, allocated 0x500 bytes into it right after the first embedded string (so that it impacts the most pointers and all), and changed all the exporter info that you see in Granny Viewer. Here is the file is you want to verify for yourself:

http://www.megaupload.com/?d=SU4X5ZSP

Open it up in Granny Viewer and my app. Check out the Exporter Info data in Granny Viewer

I couldn't find anything wrong with it. All names appeared ok in granny viewer, all the meshes rendered fine...if you can find something wrong let me know

So what does this mean? This means I can write new GR2 files. I can add/remove from existing GR2 files (via the allocater/deallocater in the sentinel class). I have much to do still on writing new bones, meshes, materials, etc. but the hard part is done - the updating of all pointers in the file

I'll work on adding new bones first. The process should be simple: allocate space in the GR2 file the size of the bone structure, add new pointers, update all pointers, add embedded string descriptors for the new embedded string describing the new bone name, update file contents, write file, done....

[TheDarkWraith]

I was feeling really bold so I said let's see how the app does on a very complex GR2 file - the NSS_Uboat7b.GR2. It's a 'special' GR2 file - it has pointers before data. It has additional data, it's a nightmare when it comes to parsing this file

I allocated 0x1000 bytes into the file after an embedded string and changed the exporter info.

The app handled it beautifully

Passed the Granny Viewer test also.

Verify for yourself: http://www.megaupload.com/?d=OGKNYJ0D


Pointers trip me up, always have and always will. They are the one thing in C++ that constantly gives me fits. That being said, the pointers in the GR2 files were tripping me up badly. I just couldn't put my head around how to update all the pointers (and all the data attached to them) - it seemed an impossible task.

I went back to programming 101: for any problem define the problem. Break problem down into steps. Break those steps into smaller steps. Keep reapeating until you get to manageable pieces/steps. The revelation came when I wrote the sentinel class. I made everything that was read from the file register with it. This way the sentinel was the master of the file. It knew if everything was read from the file or not, what blocks of the file were not read, what x block of data was and who it belonged to, etc. Once the file was read I asked sentinel to displayall for section 0 (I wrote a DOS like command shell to interface with the sentinel - a low-level utility). It spit out everything it knew about section 0. Once I could see the data as it was laid out in the file things started clicking and making sense.
Armed with how the data structures were actually laid out in the sections I started to tackle the next problem - how to update the pointers. After 3 days of writing pseudo-code I still couldn't get an algorithm together that worked. I put the GR2 editor/viewer to the side and opened up OllyDebug and started working on the CO2 bug problem again. A breakpoint triggered on a statement:

mov ESP, [EBP+EAX+offset]

Bingo!! Addressing modes. It all clicked right then. I closed OllyDebug and reopened Visual Studio. Fired up the GR2Editor/Viewer. Went and changed all my classes from using hard-coded offsets (they read offset from pointer and stored that value) to basing their offset from the pointer (kept a reference of the pointer with each class). This way everything contained in the class and referenced from the class was relative to the pointer. Problem solved

EDIT:

allocated multiple 'chunks' in the NSS_UBoat7b.GR2 file with no problems. Allocated chunks right before bone extended data, inbetween vertexdata structures, before model structure....doesn't matter where I place them nothing abnormal happens. They all pass Granny Viewer and my app (trying to reload them). Beautiful!

Now that what I thought was an impossible task has been realized I can set this aside for a day. Time to go snowboarding!!!!!!

[fromhell]

really really brilliant stuff.
your doing outstanding work,well done.

[Sartoris]

A fascinating read, as always!

[Will-Rommel]

I really enjoy the detailed explanations you go trough, because it shows a path in troubleshooting/problems resolution that might come handy for me later in my life.

[tonschk]

I Agree

Originally Posted by TheDarkWraith

for any problem define the problem. Break problem down into steps. Break those steps into smaller steps. Keep reapeating until you get to manageable pieces/steps.

Originally Posted by Will-Rommel

I really enjoy the detailed explanations you go trough, because it shows a path in troubleshooting/problems resolution that might come handy for me later in my life.

[TheDarkWraith]

Today I decided to be really bold and add extended data to the exporter info section in the GR2 file (I was going to let you see the original granny exporter version used). I wrote the necessary entries into the required section but now I'm having problems when an item goes to read it's extendeddata. Obviously I have a code bug somewhere or there's something I don't fully understand about the extended data...

[TheDarkWraith]

found a source of my problem. Every section's pointers are located in section 6 (from what I've seen so far). I've only seen pointers for Section 0 and Section 6 defined also, no other sections. I added a new datatype definition so that a new menu entry would show in Granny Viewer for the exporter info's extendeddata (you'd be able to click on it and view the extended data in granny viewer). When I added the new datatype definition I adjusted the pointers in section 6 to reflect the size of the chunk I added to that section. What I did not do was update the PointersFileOffset and EndOfPointersFileOffset in all the sections prior to section 6 Thus all those sections went looking for their pointers and was reading incorrect data. That problem solved.

Now I just have one little problem where some of section 6's section data isn't getting updated correctly. Tracking those down now and hopefully have this new menu entry ready for granny viewer here soon...

[TheDarkWraith]

This is a little funny...

I had everything coded in to show this new menu item in Granny Viewer for the exporter info. I hit Start Debugging in Visual Studio. I selected the lifeboat GR2 file. It started loading the file. Then a breakpoint was triggered...odd...new pointers found was the reason. Oh this is interesting Where did these come from? Hmm....okay let's account for them. I coded in a whole new set of pointers for this one section.
Started debugging again. Selected lifeboat GR2 file. Started loading the file. Lifeboat appeared onscreen. Cool. Alright let's go check what these new pointers were (I had sentinel spit them out to a file).
I opened them up and they looked very familiar. I checked another section and they were identical Wait a second...what's going on...
Looked in the DbgView output for both sections....yep pointer locations are different....wait...ah crap these pointers are the first sections pointers
Time to watch a movie...

[THE_MASK]

I like to see these
I dont like seeing these

[TheDarkWraith]

Finally figured out what we were calling root object type and root object offset are in the InfoHeader: they deal with the ArtToolInfo section in Granny Viewer. The first (root object type) is the section. Root object offset is the offset in that section to find the data

[Sartoris]