GR2 Editor/Viewer/Extractor/Importer - Page 33

TheDarkWraith · 01-18-12, 05:16 PM

Quote:

Originally Posted by Ekmek

sorry bump to tell what it is - here are the blender import/export scripts for obj and nif files

I've already downloaded them. I won't get to them until after I finish the current version of the app I'm working on (cloning and changing of materials/textures and possibly the ability to add new bones)

Ekmek · 01-18-12, 05:41 PM

Quote:

Originally Posted by TheDarkWraith

I've already downloaded them. I won't get to them until after I finish the current version of the app I'm working on (cloning and changing of materials/textures and possibly the ability to add new bones)

cool! Thanks!

TheDarkWraith · 01-19-12, 04:33 AM

Been working heavily on optimizing the app on the loading sequence. So far I've cut the loading time in half

I'm diving into multi-threading now...More work to do...

TheDarkWraith · 01-19-12, 02:00 PM

from first level optimizations and spawning background worker threads I've really cut down the loading times

King George V used to take 28 seconds to load on my system. Now it takes 8

NSS_Uboat7b used to take 35 seconds to load on my system. Now it takes 11

More profiling to do and more optimizations needed

TheDarkWraith · 01-19-12, 02:33 PM

Going to make another huge change to the app. I was doing the optimizations and I was removing all the unneeded reference calls by defining a static variable of that call in the class when an idea hit me. I've been grappling with the best way to define how x is related to y in the gr2 file. I gave every object a pointer and just updated these pointers. While it's functional it's very complex and quite frankly awful coding. The solution: a linked list! If you don't know what that is google it.

All my object classes inherit from one base class: GR2Base. GR2Base defines the common functionality they all need: offset in file, size, whether modified or not, and a couple methods.

Using the linked list I'll add two new variables into GR2Base: prevObject and nextObject. These will both be of type GR2Base. Since anything that inherits from GR2Base is a GR2Base object I can put any object in these.

So using the linked list we start at the beginning of the file:

FileHeader. It reads it's data. Now it knows it's own offset in the file (which is 0x0) and it knows it's own size. I set prevObject to null to signify start of the linked list.

Next comes InfoHeader. I pass in a reference to FileHeader and set prevObject with it. I also set this FileHeader reference's nextObject to this (this is a reference to the current object - the InfoHeader). Now InfoHeader can figure out it's offset by adding prevObject's offset and size together. Once it knows it's offset it reads it's data.

Now we have a linked list of two objects. I continue this pattern for all objects. Now every object is relative to another.

Now the cool part: when I want to add or remove an object I simply remove it from the chain. Since everything is relative to everything I don't have to adjust any pointers because it's automatically taken care of. Clean and simple

THE_MASK · 01-21-12, 06:44 PM

Sorry for the noob question .
Quote ''FileHeader. It reads it's data''
Could this be done with the exe ?

TheDarkWraith · 01-22-12, 12:46 AM

Quote:

Originally Posted by sober

Sorry for the noob question .
Quote ''FileHeader. It reads it's data''
Could this be done with the exe ?

Not following you. What do you mean?

The monumental recoding effort is complete

The code needed to be revamped because I've learned much more from when I first wrote it to now. Some things that were unknown or I was unsure about were resolved and coded in.

I've optimized it (only done one pass for optimizations so far - more profiling needs to be done to figure out the slow spots) and very pleased with the results. I can load the King George V in 7 seconds flat. Granny Viewer still blows me away on this but I'm getting better. The slowest part of the whole app is the reading of the verticies followed by the reading of extendeddata. That is where I will concentrate optimizing now.

I have the linked list in place code wise and items are added to it as they are read. I have more to code in for this though. I want it to tell me lots of data (whether list is contiguous, where the breaks are [and what they are], list of all x items, a visual map of the file, etc.

I have to put it through it paces to ensure I didn't break anything on the displaying of texture coordinates, importing/exporting objects, picking, and mesh transformations yet

I've got a 4.5hr flight to Portland, OR tomorrow so plenty of time to test these things!

Work continues...

Sartoris · 01-22-12, 07:56 AM

Glad to see you're making more progress with every day!

tonschk · 01-22-12, 09:12 AM

I agree 100%

Quote:

Originally Posted by DrJones

Thanks for your huge effort for this and the hard work. It is something very important for the future of sh5

Thank you very much....

Best Regards DrJones

01-22-12, 05:50 PM

A Masterful job so far Mate.

I'm nowhere near the programmer you are and what you are codeing
is what I can see and do in Hex.

But what We can do in Hex? Others would never try to duplicate.

I salute your work with a dropped jaw and the highest respects!

kapitan_zur_see · 01-23-12, 11:36 AM

I second Privateer!

This thread has come to be quite like the only one I've been reading for the past months as SH5 modding now entirely depends on it (yes, entirely).

SH5 modding has come to an almost complete stop now, as everything doable right out of the box has been done, so to speak, and its only some refining of past breakthroughs.

Nothing new will come next unless you complete your promising tool!

People with extensive knowledge of Hex editing the like of Privateer don't come too often, and the effort it requires for each new modder to reinvent the wheel simply won't makes it go anywhere...

I'm simply amazed at your skill, your willingness to share, and the dedication you're putting into this marvelous piece of software you're coding.

Without you, modding SH5 will have permanently decayed as it has already began to.

Much thanks for letting us improve the game and the possiblity to unleash the community creativity!

Sammi79 · 01-23-12, 12:23 PM

Kudos to TDW and all modders here I have been watching this thread since I uninstalled SH5 - It seems the dreams of the past are becoming reality.

I have a question, if TDW or Privateer or anyone else who knows would indulge me, it is this :

When you folks talk about 'hex' do you mean machine code? (assembly using opcodes) or am I deluding myself?

I only ask as I recently started programming assembly on a Z80 emulator (purely for intellectual satisfaction you understand I know it has very few applications in the real world) and a dream of mine was to write games back in the 8-bit days, but as a child my brain couldn't cope with more than basic. and I am lately having success in that meager ambition.

yeah retro I know, but it is really good fun if you're me!

regards,
Sam.

01-23-12, 12:40 PM

Working in Hex looks like this.

The thing is, you need to figure out what all the stuff means!

Sammi79 · 01-23-12, 12:48 PM

OK thanks Privateer, that is exactly what assembled machine code looks like, using hex numbers for ease rather than binary, each byte containing an 8-bit number represented in hex. I can see 'words' - N o r m a l - I assume they are strings for text passing, and the garbage in between is the opcodes/operands? or am I wandering down a dark alley here?

PS sorry TDW to off topic your thread like this, If it is a problem I shall zip it.

regards,
Sam.

01-23-12, 01:17 PM

The in between stuff is what We have to figure out.
(That image is a random one from the 'Net)
Looking at an SH5 GR2 file in Hex and in GrannyViewer,
one can figure out what all this mess is.
Some are pointers, some are information of different kinds, etc.

It's the studying of files and file structures that throws people.
To most? It's a big mess of nothing in Enigma code!
To a few? It's all the information We need. We just have to learn what it all means.

01-23-12, 11:36 AM	#491
kapitan_zur_see Commodore Join Date: Mar 2005 Location: France Posts: 614 Downloads: 60 Uploads: 0	I second Privateer! This thread has come to be quite like the only one I've been reading for the past months as SH5 modding now entirely depends on it (yes, entirely). SH5 modding has come to an almost complete stop now, as everything doable right out of the box has been done, so to speak, and its only some refining of past breakthroughs. Nothing new will come next unless you complete your promising tool! People with extensive knowledge of Hex editing the like of Privateer don't come too often, and the effort it requires for each new modder to reinvent the wheel simply won't makes it go anywhere... I'm simply amazed at your skill, your willingness to share, and the dedication you're putting into this marvelous piece of software you're coding. Without you, modding SH5 will have permanently decayed as it has already began to. Much thanks for letting us improve the game and the possiblity to unleash the community creativity! __________________

01-23-12, 12:23 PM	#492
Sammi79 XO Join Date: Jan 2010 Location: Penzance Posts: 428 Downloads: 272 Uploads: 0	Kudos to TDW and all modders here I have been watching this thread since I uninstalled SH5 - It seems the dreams of the past are becoming reality. I have a question, if TDW or Privateer or anyone else who knows would indulge me, it is this : When you folks talk about 'hex' do you mean machine code? (assembly using opcodes) or am I deluding myself? I only ask as I recently started programming assembly on a Z80 emulator (purely for intellectual satisfaction you understand I know it has very few applications in the real world) and a dream of mine was to write games back in the 8-bit days, but as a child my brain couldn't cope with more than basic. and I am lately having success in that meager ambition. yeah retro I know, but it is really good fun if you're me! regards, Sam. __________________ Gadewais fy beic nghadwyno i'r rhai a rheiliau, pan wnes i ddychwelyd, yno mae'n roedd... Wedi mynd.

01-23-12, 12:48 PM	#494
Sammi79 XO Join Date: Jan 2010 Location: Penzance Posts: 428 Downloads: 272 Uploads: 0	OK thanks Privateer, that is exactly what assembled machine code looks like, using hex numbers for ease rather than binary, each byte containing an 8-bit number represented in hex. I can see 'words' - N o r m a l - I assume they are strings for text passing, and the garbage in between is the opcodes/operands? or am I wandering down a dark alley here? PS sorry TDW to off topic your thread like this, If it is a problem I shall zip it. regards, Sam. __________________ Gadewais fy beic nghadwyno i'r rhai a rheiliau, pan wnes i ddychwelyd, yno mae'n roedd... Wedi mynd.

01-19-12, 04:33 AM	#483
TheDarkWraith Black Magic Join Date: Jun 2007 Posts: 11,962 Downloads: 147 Uploads: 5	Been working heavily on optimizing the app on the loading sequence. So far I've cut the loading time in half I'm diving into multi-threading now...More work to do...

01-19-12, 02:00 PM	#484
TheDarkWraith Black Magic Join Date: Jun 2007 Posts: 11,962 Downloads: 147 Uploads: 5	from first level optimizations and spawning background worker threads I've really cut down the loading times King George V used to take 28 seconds to load on my system. Now it takes 8 NSS_Uboat7b used to take 35 seconds to load on my system. Now it takes 11 More profiling to do and more optimizations needed

01-19-12, 02:33 PM	#485
TheDarkWraith Black Magic Join Date: Jun 2007 Posts: 11,962 Downloads: 147 Uploads: 5	Going to make another huge change to the app. I was doing the optimizations and I was removing all the unneeded reference calls by defining a static variable of that call in the class when an idea hit me. I've been grappling with the best way to define how x is related to y in the gr2 file. I gave every object a pointer and just updated these pointers. While it's functional it's very complex and quite frankly awful coding. The solution: a linked list! If you don't know what that is google it. All my object classes inherit from one base class: GR2Base. GR2Base defines the common functionality they all need: offset in file, size, whether modified or not, and a couple methods. Using the linked list I'll add two new variables into GR2Base: prevObject and nextObject. These will both be of type GR2Base. Since anything that inherits from GR2Base is a GR2Base object I can put any object in these. So using the linked list we start at the beginning of the file: FileHeader. It reads it's data. Now it knows it's own offset in the file (which is 0x0) and it knows it's own size. I set prevObject to null to signify start of the linked list. Next comes InfoHeader. I pass in a reference to FileHeader and set prevObject with it. I also set this FileHeader reference's nextObject to this (this is a reference to the current object - the InfoHeader). Now InfoHeader can figure out it's offset by adding prevObject's offset and size together. Once it knows it's offset it reads it's data. Now we have a linked list of two objects. I continue this pattern for all objects. Now every object is relative to another. Now the cool part: when I want to add or remove an object I simply remove it from the chain. Since everything is relative to everything I don't have to adjust any pointers because it's automatically taken care of. Clean and simple

01-21-12, 06:44 PM	#486
THE_MASK Ace of the deep . Join Date: Jan 2006 Posts: 9,226 Downloads: 901 Uploads: 73	Sorry for the noob question . Quote ''FileHeader. It reads it's data'' Could this be done with the exe ?

01-22-12, 07:56 AM	#488
Sartoris Captain Join Date: May 2011 Posts: 489 Downloads: 106 Uploads: 0	Glad to see you're making more progress with every day!

01-22-12, 05:50 PM	#490
Madox58 Stowaway Posts: n/a Downloads: Uploads:	A Masterful job so far Mate. I'm nowhere near the programmer you are and what you are codeing is what I can see and do in Hex. But what We can do in Hex? Others would never try to duplicate. I salute your work with a dropped jaw and the highest respects!

01-23-12, 12:40 PM	#493
Madox58 Stowaway Posts: n/a Downloads: Uploads:	Working in Hex looks like this. The thing is, you need to figure out what all the stuff means!

01-23-12, 01:17 PM	#495
Madox58 Stowaway Posts: n/a Downloads: Uploads:	The in between stuff is what We have to figure out. (That image is a random one from the 'Net) Looking at an SH5 GR2 file in Hex and in GrannyViewer, one can figure out what all this mess is. Some are pointers, some are information of different kinds, etc. It's the studying of files and file structures that throws people. To most? It's a big mess of nothing in Enigma code! To a few? It's all the information We need. We just have to learn what it all means.