GR2 Editor/Viewer/Extractor/Importer - Page 31

Ekmek · 01-16-12, 04:03 PM

Quote:

Originally Posted by TheDarkWraith

Yes you will get that error on non-SH5 GR2 files currently. I have it 'locked' to only SH5 GR2 file format for the time being. If I can get everything working with SH5 files then making it work with everything else should be easier. Key word being should.

Its great to see your progress. Out of curiousity how hard would it be to make an import/export script for Blender? I figure this might be the quickest way to get all the tools you are looking for. If you are not familar with a blender script I can post one.

Thank you for all your hard work.

TheDarkWraith · 01-16-12, 04:06 PM

I thought that adding a new menu item to Granny Viewer would be a relatively easy task.....not even. I still don't understand how the Granny Viewer knows from the pointers that an item has extended data. I haven't been able to find a smoking gun yet. Without this smoking gun I can't add new bones, materials, etc.

I have tons of data on the pointers and now from the data descriptors for the Granny Viewer menu items contained in the GR2 file. Now it's just a matter of sifting through it all and figuring out what it all means.

I did make one discovery and I couldn't believe it was staring me in the face all along

I wondered exactly how the GR2 file was laid out for the longest time. I wrote the sentinel class to tell me this info. After coorelating the data from it with the Granny Viewer it was quite evident that, once again, Granny Viewer tells you everything you need to know. Right there on that first screen when you select view in detail is how the file is laid out

Now if I can just figure out how it knows an item has extended data...

TheDarkWraith · 01-16-12, 04:08 PM

Quote:

Originally Posted by Ekmek

Its great to see your progress. Out of curiousity how hard would it be to make an import/export script for Blender? I figure this might be the quickest way to get all the tools you are looking for. If you are not familar with a blender script I can post one.

Thank you for all your hard work.

I'm not familiar with the Blender format. If you can give me documentation on it then I can see what I can do

TheBeast · 01-16-12, 04:23 PM

Quote:

Originally Posted by TheDarkWraith

I thought that adding a new menu item to Granny Viewer would be a relatively easy task.....not even. I still don't understand how the Granny Viewer knows from the pointers that an item has extended data. I haven't been able to find a smoking gun yet. Without this smoking gun I can't add new bones, materials, etc.

I have tons of data on the pointers and now from the data descriptors for the Granny Viewer menu items contained in the GR2 file. Now it's just a matter of sifting through it all and figuring out what it all means.

I did make one discovery and I couldn't believe it was staring me in the face all along

I wondered exactly how the GR2 file was laid out for the longest time. I wrote the sentinel class to tell me this info. After coorelating the data from it with the Granny Viewer it was quite evident that, once again, Granny Viewer tells you everything you need to know. Right there on that first screen when you select view in detail is how the file is laid out

Now if I can just figure out how it knows an item has extended data...

So everything is not based on Node ID/Parent Node ID system?

If Node/Data does not have Parent ID does it apply to entire Model?

If Node/Data does not have Parent ID in current loaded Model, can Parent ID be in another attaching Model?

i.e. Conning_7B_01_FX.GR2 file adding Nodes for FX placement on Type-VIIB Conning Towers.

Skwas S3D did not edit all data present in Models for SHIII/SHIV.

TheDarkWraith · 01-16-12, 04:52 PM

Quote:

Originally Posted by TheBeast

So everything is not based on Node ID/Parent Node ID system?

If Node/Data does not have Parent ID does it apply to entire Model?

If Node/Data does not have Parent ID in current loaded Model, can Parent ID be in another attaching Model?

i.e. Conning_7B_01_FX.GR2 file adding Nodes for FX placement on Type-VIIB Conning Towers.

Skwas S3D did not edit all data present in Models for SHIII/SHIV.

In a way yes for the 3D models. The ID is generated from the name of the object. There is no ID per se in the GR2 files.

For these granny viewer items you have pointers that describe the data. You also have what I call datatype definitions that define the menu items seen in the Granny Viewer. I can find all the ExtendedData menu items from what sentinel spit out about the datatype definitions but I cannot correlate how it knows that an item has extended data. There is no pointer or datatype definition that defines the actual data to show. I'm beginning to think it's from just the file layout.
The Granny Viewer knows where everything is in the file from the pointers. It knows how to build the menu items seen in it from the datatype definitions. Now if it knows that object A is at position X and object B is at position Y and object A's position + A's size != object B's position then it must have extended data defined. That's what I'm thinking is going on with the extended data...

TheDarkWraith · 01-16-12, 05:52 PM

The more I work with this Granny system the more I like it

Once you understand it (and I'm nowhere near close to fully understanding it) it all makes perfect sense and it's quite logical, it's actually quite brilliant

I took the data from sentinel on the datatype definitions and mapped it out on paper. I had the main menu from Granny Viewer (the one that shows when you right click and select view in detail) drawn out. I had branches from each menu item drawn to it's next node. I had branches from those nodes to their sub nodes. So on and so forth. Now I had a perfect tree that shows a) what the data looks like in those datatype definitions and b) how they all went together. Once I studied this tree it was quite evident that each node (and it's children) describe a 'page' in Granny Viewer (when you click on something). These entries define the entries to show when the item is clicked. Now whether those items have data associated with them is not located here. These are just the blueprint on how to make the menu entries. Good I understand that and it makes sense.

Now I should say that there is more data contained in these datatype definitions than just how to make a menu item. It also describes the type of data that the menu item requires. This is needed because once something is pointed to the start of a blueprint, the whole blueprint is read until what I call an end of data type definition record is read. Once that record is encountered then it knows that it doesn't require anymore data. Now as the blueprint is being read, the data type associated with the blueprint is also being read.

Think of the datatype definitions as XML in non-human readable form (Granny hex code). Sentinel converted the Granny hex code into human readable form so I could interpret it. I think I've spent more time and code writing code to interpret data than to actually use/modify it.

I went back and looked at my classes on how they read extended data. There are two pointers located at the end of every granny object - a pointer to a datatype definition and a pointer to a pointer (this pointer points to data). Just follow along. The first pointer - the datatype definition pointer - points to the start of the 'blueprint' - how Granny Viewer should display the extended data (what all the entries are). Now remember I said the blueprint doesn't define the data - it defines the menu entries to show in Granny Viewer. Okay so first pointer tells granny viewer what to display on screen when you click extended data for the object. Where does the data come from? That's where the second pointer shines. The second pointer points to where Granny Viewer will read the data from to fill in those menu items. Now in that data can be references to other pointers (or even null pointers - no data). That is why some items in granny viewer are not clickable - they have no data associated with them.

Brilliant, absolutely brilliant

IF you can't see why this is brilliant it's because data is not duplicated. If two items have the same extended data then all you have to do it point their first pointer to the same blueprint. Point their second pointers to their respective data.

Now I know how to add menu items to Granny Viewer

See this whole thing is just one big puzzle....every piece you add furthers your understanding of the whole. I love puzzles.

TheDarkWraith · 01-17-12, 01:26 AM

I think this is the first time Granny Viewer has ever seen extended data for the Exporter Info:

Clicking the extended data locks up Granny Viewer so obviously there's something else I'm missing but it's a start. I can add menu items to a GR2 file. Now I just have to figure out this last little detail

I do believe that this is the first time an application has made changes to a GR2 file that didn't use ANY of RAD game tools SDK or tools or any exporter for 3DSMax or the likes

Ekmek · 01-17-12, 01:32 AM

Quote:

Originally Posted by TheDarkWraith

I'm not familiar with the Blender format. If you can give me documentation on it then I can see what I can do

cool, thanks! If I sent you some of the import/export python scripts for obj, nif, and 3ds files would that help? I would post them now (combined its only 110kb) but I dont have priviledges to post attachments yet

TheDarkWraith · 01-17-12, 01:34 AM

Quote:

Originally Posted by Ekmek

cool, thanks! If I sent you some of the import/export python scripts for obj, nif, and 3ds files would that help? I would post them now (combined its only 110kb) but I dont have priviledges to post attachments yet

zip them up and upload to www.megaupload.com Then just send me a link to it

tonschk · 01-17-12, 09:22 AM

Me too

Quote:

Originally Posted by TheDarkWraith

I love puzzles.

TheDarkWraith · 01-17-12, 12:50 PM

Got the new menu item to show in Granny Viewer

When you click extended data for exporter info a new screen appears. There is currently one entry: Original Exporter. It's value is null because I'm missing a pointer but at least I got it to work

reaper7 · 01-17-12, 01:23 PM

Well Done TDW, Determination is your greatest tool

Vanilla · 01-17-12, 03:53 PM

When I tried to decypher gr2 in hex-editor I thought it would be plain and easy but now seeing your progress and reading about all the file structure intricacies and your clever insights on how to untwine it all.. oh mate am I impressed by your determination!

TheDarkWraith · 01-17-12, 04:25 PM

I can add new menu items now without problems. I added extended data to the exporter info to have it show the original exporter since I replaced it with mine:

The problem I had before when I said granny viewer showed the extended data screen and the new menu item but null for the value was I didn't create an extended data section in the GR2 file for the exporter info. Then I had to link this section to the granny viewer data so it knew where to find it. I knew it was something simple (and I did say I was missing a pointer)

I still decode GR2 file with hex editor. You have to. I just write tools inside the GR2 editor/viewer to decipher this hex code into human readable form. I'm not lying when I said I've spent more time writing code that deciphers data/interprets data than writing code to actually use it

The sentinel class is the interface to these tools inside the app.

The sentinel class is just awesome, one of the coolest things I've made. The way you interface with this class is through a DOS like command window. If you like DOS this is right up your alley. Sentinel has full control over the file. You ask it for help (h or help) and it shows you the commands it 'knows'. Some of the commands require a password level. Those commands that require a password level are the really low-level commands that are allowed to slice and dice the GR2 file. Alas you all won't be getting that password, sorry. Everything you all will need to do will be done using the app itself and it's interface. This sentinel is more for me to see what's going on, what data is where, what data is contained in x block of the file, etc.

If you'd like to verify for yourself or try and figure out how I did it, here you go: http://www.megaupload.com/?d=0YIPC2Z1

EDIT:

now my thinking is to encode data into the GR2 file pertaining to when the file was last updated (date and time), who updated it, who the original author is, etc. What data would you like to see encoded into the GR2 file for when you're making new ones?

TheDarkWraith · 01-17-12, 04:48 PM

As it stands now the current version of the app will allow you to clone a GR2 file and rename anything you want in it (you could also replace the geometry with new geometry). This is great because you can then assign new textures to the clone to make new units (via new names). Aircraft are not be to cloned yet as I have to figure out all this trackgroup crap.

So I will concentrate on implementing this cloning ability and release an updated version so you all can start cloning units

01-16-12, 05:52 PM	#456
TheDarkWraith Black Magic Join Date: Jun 2007 Posts: 11,962 Downloads: 147 Uploads: 5	The more I work with this Granny system the more I like it Once you understand it (and I'm nowhere near close to fully understanding it) it all makes perfect sense and it's quite logical, it's actually quite brilliant I took the data from sentinel on the datatype definitions and mapped it out on paper. I had the main menu from Granny Viewer (the one that shows when you right click and select view in detail) drawn out. I had branches from each menu item drawn to it's next node. I had branches from those nodes to their sub nodes. So on and so forth. Now I had a perfect tree that shows a) what the data looks like in those datatype definitions and b) how they all went together. Once I studied this tree it was quite evident that each node (and it's children) describe a 'page' in Granny Viewer (when you click on something). These entries define the entries to show when the item is clicked. Now whether those items have data associated with them is not located here. These are just the blueprint on how to make the menu entries. Good I understand that and it makes sense. Now I should say that there is more data contained in these datatype definitions than just how to make a menu item. It also describes the type of data that the menu item requires. This is needed because once something is pointed to the start of a blueprint, the whole blueprint is read until what I call an end of data type definition record is read. Once that record is encountered then it knows that it doesn't require anymore data. Now as the blueprint is being read, the data type associated with the blueprint is also being read. Think of the datatype definitions as XML in non-human readable form (Granny hex code). Sentinel converted the Granny hex code into human readable form so I could interpret it. I think I've spent more time and code writing code to interpret data than to actually use/modify it. I went back and looked at my classes on how they read extended data. There are two pointers located at the end of every granny object - a pointer to a datatype definition and a pointer to a pointer (this pointer points to data). Just follow along. The first pointer - the datatype definition pointer - points to the start of the 'blueprint' - how Granny Viewer should display the extended data (what all the entries are). Now remember I said the blueprint doesn't define the data - it defines the menu entries to show in Granny Viewer. Okay so first pointer tells granny viewer what to display on screen when you click extended data for the object. Where does the data come from? That's where the second pointer shines. The second pointer points to where Granny Viewer will read the data from to fill in those menu items. Now in that data can be references to other pointers (or even null pointers - no data). That is why some items in granny viewer are not clickable - they have no data associated with them. Brilliant, absolutely brilliant IF you can't see why this is brilliant it's because data is not duplicated. If two items have the same extended data then all you have to do it point their first pointer to the same blueprint. Point their second pointers to their respective data. Now I know how to add menu items to Granny Viewer See this whole thing is just one big puzzle....every piece you add furthers your understanding of the whole. I love puzzles. Last edited by TheDarkWraith; 01-16-12 at 06:19 PM.

01-16-12, 04:06 PM	#452
TheDarkWraith Black Magic Join Date: Jun 2007 Posts: 11,962 Downloads: 147 Uploads: 5	I thought that adding a new menu item to Granny Viewer would be a relatively easy task.....not even. I still don't understand how the Granny Viewer knows from the pointers that an item has extended data. I haven't been able to find a smoking gun yet. Without this smoking gun I can't add new bones, materials, etc. I have tons of data on the pointers and now from the data descriptors for the Granny Viewer menu items contained in the GR2 file. Now it's just a matter of sifting through it all and figuring out what it all means. I did make one discovery and I couldn't believe it was staring me in the face all along I wondered exactly how the GR2 file was laid out for the longest time. I wrote the sentinel class to tell me this info. After coorelating the data from it with the Granny Viewer it was quite evident that, once again, Granny Viewer tells you everything you need to know. Right there on that first screen when you select view in detail is how the file is laid out Now if I can just figure out how it knows an item has extended data...

01-17-12, 01:26 AM	#457
TheDarkWraith Black Magic Join Date: Jun 2007 Posts: 11,962 Downloads: 147 Uploads: 5	I think this is the first time Granny Viewer has ever seen extended data for the Exporter Info: Clicking the extended data locks up Granny Viewer so obviously there's something else I'm missing but it's a start. I can add menu items to a GR2 file. Now I just have to figure out this last little detail I do believe that this is the first time an application has made changes to a GR2 file that didn't use ANY of RAD game tools SDK or tools or any exporter for 3DSMax or the likes

01-17-12, 12:50 PM	#461
TheDarkWraith Black Magic Join Date: Jun 2007 Posts: 11,962 Downloads: 147 Uploads: 5	Got the new menu item to show in Granny Viewer When you click extended data for exporter info a new screen appears. There is currently one entry: Original Exporter. It's value is null because I'm missing a pointer but at least I got it to work

01-17-12, 01:23 PM	#462
reaper7 sim2reality Join Date: Jun 2007 Location: AM 82 Posts: 2,280 Downloads: 258 Uploads: 30	Well Done TDW, Determination is your greatest tool

01-17-12, 03:53 PM	#463
Vanilla Lieutenant Join Date: Nov 2006 Location: St. Petersburg, Russia Posts: 264 Downloads: 72 Uploads: 0	When I tried to decypher gr2 in hex-editor I thought it would be plain and easy but now seeing your progress and reading about all the file structure intricacies and your clever insights on how to untwine it all.. oh mate am I impressed by your determination!

01-17-12, 04:25 PM	#464
TheDarkWraith Black Magic Join Date: Jun 2007 Posts: 11,962 Downloads: 147 Uploads: 5	I can add new menu items now without problems. I added extended data to the exporter info to have it show the original exporter since I replaced it with mine: The problem I had before when I said granny viewer showed the extended data screen and the new menu item but null for the value was I didn't create an extended data section in the GR2 file for the exporter info. Then I had to link this section to the granny viewer data so it knew where to find it. I knew it was something simple (and I did say I was missing a pointer) I still decode GR2 file with hex editor. You have to. I just write tools inside the GR2 editor/viewer to decipher this hex code into human readable form. I'm not lying when I said I've spent more time writing code that deciphers data/interprets data than writing code to actually use it The sentinel class is the interface to these tools inside the app. The sentinel class is just awesome, one of the coolest things I've made. The way you interface with this class is through a DOS like command window. If you like DOS this is right up your alley. Sentinel has full control over the file. You ask it for help (h or help) and it shows you the commands it 'knows'. Some of the commands require a password level. Those commands that require a password level are the really low-level commands that are allowed to slice and dice the GR2 file. Alas you all won't be getting that password, sorry. Everything you all will need to do will be done using the app itself and it's interface. This sentinel is more for me to see what's going on, what data is where, what data is contained in x block of the file, etc. If you'd like to verify for yourself or try and figure out how I did it, here you go: http://www.megaupload.com/?d=0YIPC2Z1 EDIT: now my thinking is to encode data into the GR2 file pertaining to when the file was last updated (date and time), who updated it, who the original author is, etc. What data would you like to see encoded into the GR2 file for when you're making new ones?

01-17-12, 04:48 PM	#465
TheDarkWraith Black Magic Join Date: Jun 2007 Posts: 11,962 Downloads: 147 Uploads: 5	As it stands now the current version of the app will allow you to clone a GR2 file and rename anything you want in it (you could also replace the geometry with new geometry). This is great because you can then assign new textures to the clone to make new units (via new names). Aircraft are not be to cloned yet as I have to figure out all this trackgroup crap. So I will concentrate on implementing this cloning ability and release an updated version so you all can start cloning units