Open source tool(s) for SH5

TheDarkWraith · 10-15-09, 07:27 PM

I've been reading that SH5 will be similar in file structure and all to SH3/4 which is good but also bad. Good because some things will readily stand out and be familiar in the hex code. Bad because until a new tool, or revised tool (tool being an application or applications), is made most will not be able to take advantage of whatever the designers decided to 'give' us. Some of us will be able to play around with the new stuff because we can 'read' the hex code and know how to manipulate it. But as a whole that doesn't do anybody any good except those people (unless they divulge their findings to others which rarely happens).
People have made applications to help them mod and understand SH3/4, I being one of them. I haven't made my applications public (only to a select few) because I was the sole designer in those applications. It's hard to 'let go' of your prized possessions when all the work was done by yourself. Plus to try and write a help file or to have to constantly explain why it does this or it does that was not something I was willing to deal with. I knew how my application worked because I wrote it (and if I forgot I just had to look at the code again). With open source, everyone knows how everything works.
Thus I am thinking about spear heading a new open source project for SH5. Most of you use S3D (I do not) and since I've read that it is no longer being developed the application will probably be limited in what one can achieve with the files in SH5 (I'm merely speculating here as noone will know until we see the files in SH5). Furthermore, because the application was based on company code, the source code can't be released for others to update/maintain (but since it's MIL code it IS able to be reversed engineered somewhat easily

). Herein lies the beauty in open source code.
I know there are others here who write code. Maybe your strength is C++, C#, J#, or maybe even Assembly (love it!!) or maybe you like to dabble and experiment. With Microsoft's (yes, the evil empire) .NET framework and use of MIL code, modules and .dlls can be written in any language (available to Visual Studio or can be compiled to MIL - inline Assembly is also supported in Visual Studio

). Maybe you can't write code but you're an expert in linear algebra (many uses in DirectX) or graphics design or have useful knowledge about the game files. There are uses for many different kinds of specialties in application development.
Since the 'files' will be made public, source code and all, they can be fully debugged and refined to perfection. With many eyes looking over them (files) features never thought of can be discovered. Maybe you have an idea for a feature but don't know how to implement it? Open source is your answer! Some people will take your idea and 'make' it just to see if they can do it (I'm that way).
Right now I'm looking to see if there's an interest in this. With community involvement I imagine a tool can be made that would even make the designers of the game take a look and notice.
This idea hit me as I was looking over my applications that I will most likely have to change to work with SH5. Instead of revamping them all I thought about making an entirely new one. Something better. It doesn't have to be just one application...maybe specialized tools or a set of RAD (Rapid Application Development) tools to test an idea or to try and understand something.

One might ask, "A bit early to start on an ambitious project like this don't you think? I mean the game hasn't even been released yet." I do not think so. Why? Many reasons:

Ideas and information start being exchanged
ground work can be laid (coding structure, variables, functions, use of indentation, etc.)
Specialty tools can be designed and implemented from information gained in 1 on existing files. Example: a file reader. Design a generic file reader that given a list of structures will parse the file. Have the list of structures be read from a file so that new structures can be added as they are discovered/learned. This would allow the addition of new nodes as they are discovered in SH5 (if there are any new ones).
people become 'up to speed' on the file structure and use of controllers through design of tools
a set of RAD tools or .dlls can be designed to make the design of new tools faster and easier
Individuals can use the .dlls to make their own applications to help them understand on their own. Example: using the file reader one could extract the 3D model data and render the object. Then an application could be developed to add/remove nodes easily (by using visual objects - one of my applications does this). Instead of coding new nodes you simply open up an object that has the object you want and drag it over to your object. Position and rotate where it needs to be and presto...it's done.

When a team assembles and information and ideas start flowing the impossible doesn't look so impossible anymore. Everything starts with an idea

kapitan_zur_see · 10-15-09, 07:56 PM

I have but no coding knowledge, however I'm into graphic design and yes, some ideas are always good to share! I'm definately into your opensource project! Key thing is that modders must not feel depossessed from their work/ have the feeling of having their work not solely credited etc. However, it would certainly help A Lot to code a new multipurpose modding tool by not being a "one guy only" task. Way faster probably

Totally second you, as I've always been keen on sharing knowledge back when me and leovampire (rest his soul, we were learning the basis of sh4 modding together and improved each other skills this way

TheDarkWraith · 10-15-09, 08:27 PM

the idea here is not to create mods...the idea is to create tools to help others create mods. To create tools to help understand and learn. To create tools to do things people have thought of but don't know how to implement. To create reusable modules that can be incorporated in other projects - OOP at it's best!

Anything created is in the name of open source and not the creator(s).
Maybe someone has an application that can extract data from game files but because they lack the knowledge in DirectX they can't 'display' the data onscreen....maybe someone else has been trying to do what the former's application can do but they can't seem to code the file reading BUT they can 'render' in DirectX no problem....a perfect matchup. With a set groundwork of rules (how variables are named, indentation, function use, etc.) they can help each other out.
In open source no one is the owner of the 'project'. Is it owned by everyone. There would be a manager of the project but that's about it.

The first thing to do is get a team together and hash out the 'rules' and guidelines. Once the groundwork is laid and everyone understands it it's just a matter of writing code, sharing code, sharing ideas, and learning. Every good software application is planned before the first bit of code is ever written. And usually that plan is updated during the course of development because you cannot possibly think of everything. I'm sure if there is enough interest that we can get our own forum on here....if not, there's always BTS mods or some other place to gather.

Say you're using your favorite tool and you wish it could do this or it could do that. If it's open source code then you can make whatever changes you like since you have the source code.

It would be neat to be able to add the use of scripting to modules and such. I'm learning Python for the upcoming SH5 release but haven't learned how to integrate it with C++ or C# yet.

THE_MASK · 10-16-09, 01:41 AM

It would be good if as a modder you could use this program to make more human figures and animations . http://www.gamedev.net/features/revi...?productid=762

kapitan_zur_see · 10-16-09, 04:42 PM

Quote:

Originally Posted by Racerboy

the idea here is not to create mods...the idea is to create tools to help others create mods. To create tools to help understand and learn.

I got your point already on that

I was rather talking about people who mods a thing, discovering a new process of altering some of the game behaviour or implementation and won't share the knowledge to other modders and especially devs of such tools who have to figure out everything by themselves

kapitan_zur_see · 10-16-09, 04:43 PM

Quote:

Originally Posted by sober

It would be good if as a modder you could use this program to make more human figures and animations . http://www.gamedev.net/features/revi...?productid=762

problem here will be how animations are implemented ingame... I never succed to figure out how to script an animation in SH4, poser or 3dsmax, anything...

urfisch · 10-16-09, 06:27 PM

i think the idea is great! even i could not help, i am very happy to see racerboy sweepin round here again...

10-16-09, 06:38 PM

Quote:

Originally Posted by sober

It would be good if as a modder you could use this program to make more human figures and animations . http://www.gamedev.net/features/revi...?productid=762

if thats true then the first thing i'll do with it is create gilligan's island and put the castaways on it

skwasjer · 10-17-09, 02:05 PM

I've left the Dat.dll in S3D on purpose unobfuscated and even included source code for all controllers, just so anyone can (re)use it. I've always encouraged open source for that matter. The problem is the underlying framework is copyright protected so I can't release all of that. I can provide info/insights or examples though, no problem. S3D's file parsers are very flexible and robust by now and so it would be stupid to redesign it. I can help here too.

It will all depend on how different or not SH5 will be, and how much the devs provide info on the new storage structures. I've already did some enquiries but sadly no response. But I promised myself if I have to ever use a hexeditor again on SH-data, then you will not see another update to S3D... The amount of time it takes is disproportional, and yet many people just take it for granted.

Good luck on the project. Speaking from personal experience, you will need it. You have many challenges ahead of you...

TheDarkWraith · 11-17-09, 12:38 PM

Had to leave country for awhile on assignment but I'm back in the US again. While I was away I decided it would be better to re-design my Apollo application for the upcoming SH5 release to make it more 'modular' and easier to upgrade/modify. Here is the source code for an old version of Apollo in one of it's design stages:

http://rapidshare.com/files/30839125...nt_old.7z.html

Code is written in C# using Visual Studio 2008.

Working on new version currently (in C#) and plan on releasing source code for it once it's in a stable state.

10-15-09, 07:27 PM	#1
TheDarkWraith Black Magic Join Date: Jun 2007 Posts: 11,962 Downloads: 147 Uploads: 5	Open source tool(s) for SH5 I've been reading that SH5 will be similar in file structure and all to SH3/4 which is good but also bad. Good because some things will readily stand out and be familiar in the hex code. Bad because until a new tool, or revised tool (tool being an application or applications), is made most will not be able to take advantage of whatever the designers decided to 'give' us. Some of us will be able to play around with the new stuff because we can 'read' the hex code and know how to manipulate it. But as a whole that doesn't do anybody any good except those people (unless they divulge their findings to others which rarely happens). People have made applications to help them mod and understand SH3/4, I being one of them. I haven't made my applications public (only to a select few) because I was the sole designer in those applications. It's hard to 'let go' of your prized possessions when all the work was done by yourself. Plus to try and write a help file or to have to constantly explain why it does this or it does that was not something I was willing to deal with. I knew how my application worked because I wrote it (and if I forgot I just had to look at the code again). With open source, everyone knows how everything works. Thus I am thinking about spear heading a new open source project for SH5. Most of you use S3D (I do not) and since I've read that it is no longer being developed the application will probably be limited in what one can achieve with the files in SH5 (I'm merely speculating here as noone will know until we see the files in SH5). Furthermore, because the application was based on company code, the source code can't be released for others to update/maintain (but since it's MIL code it IS able to be reversed engineered somewhat easily ). Herein lies the beauty in open source code. I know there are others here who write code. Maybe your strength is C++, C#, J#, or maybe even Assembly (love it!!) or maybe you like to dabble and experiment. With Microsoft's (yes, the evil empire) .NET framework and use of MIL code, modules and .dlls can be written in any language (available to Visual Studio or can be compiled to MIL - inline Assembly is also supported in Visual Studio ). Maybe you can't write code but you're an expert in linear algebra (many uses in DirectX) or graphics design or have useful knowledge about the game files. There are uses for many different kinds of specialties in application development. Since the 'files' will be made public, source code and all, they can be fully debugged and refined to perfection. With many eyes looking over them (files) features never thought of can be discovered. Maybe you have an idea for a feature but don't know how to implement it? Open source is your answer! Some people will take your idea and 'make' it just to see if they can do it (I'm that way). Right now I'm looking to see if there's an interest in this. With community involvement I imagine a tool can be made that would even make the designers of the game take a look and notice. This idea hit me as I was looking over my applications that I will most likely have to change to work with SH5. Instead of revamping them all I thought about making an entirely new one. Something better. It doesn't have to be just one application...maybe specialized tools or a set of RAD (Rapid Application Development) tools to test an idea or to try and understand something. One might ask, "A bit early to start on an ambitious project like this don't you think? I mean the game hasn't even been released yet." I do not think so. Why? Many reasons: Ideas and information start being exchanged ground work can be laid (coding structure, variables, functions, use of indentation, etc.) Specialty tools can be designed and implemented from information gained in 1 on existing files. Example: a file reader. Design a generic file reader that given a list of structures will parse the file. Have the list of structures be read from a file so that new structures can be added as they are discovered/learned. This would allow the addition of new nodes as they are discovered in SH5 (if there are any new ones). people become 'up to speed' on the file structure and use of controllers through design of tools a set of RAD tools or .dlls can be designed to make the design of new tools faster and easier Individuals can use the .dlls to make their own applications to help them understand on their own. Example: using the file reader one could extract the 3D model data and render the object. Then an application could be developed to add/remove nodes easily (by using visual objects - one of my applications does this). Instead of coding new nodes you simply open up an object that has the object you want and drag it over to your object. Position and rotate where it needs to be and presto...it's done. When a team assembles and information and ideas start flowing the impossible doesn't look so impossible anymore. Everything starts with an idea Last edited by TheDarkWraith; 10-15-09 at 09:40 PM.

10-15-09, 07:56 PM	#2
kapitan_zur_see Commodore Join Date: Mar 2005 Location: France Posts: 614 Downloads: 60 Uploads: 0	I have but no coding knowledge, however I'm into graphic design and yes, some ideas are always good to share! I'm definately into your opensource project! Key thing is that modders must not feel depossessed from their work/ have the feeling of having their work not solely credited etc. However, it would certainly help A Lot to code a new multipurpose modding tool by not being a "one guy only" task. Way faster probably Totally second you, as I've always been keen on sharing knowledge back when me and leovampire (rest his soul, we were learning the basis of sh4 modding together and improved each other skills this way __________________

10-15-09, 08:27 PM	#3
TheDarkWraith Black Magic Join Date: Jun 2007 Posts: 11,962 Downloads: 147 Uploads: 5	the idea here is not to create mods...the idea is to create tools to help others create mods. To create tools to help understand and learn. To create tools to do things people have thought of but don't know how to implement. To create reusable modules that can be incorporated in other projects - OOP at it's best! Anything created is in the name of open source and not the creator(s). Maybe someone has an application that can extract data from game files but because they lack the knowledge in DirectX they can't 'display' the data onscreen....maybe someone else has been trying to do what the former's application can do but they can't seem to code the file reading BUT they can 'render' in DirectX no problem....a perfect matchup. With a set groundwork of rules (how variables are named, indentation, function use, etc.) they can help each other out. In open source no one is the owner of the 'project'. Is it owned by everyone. There would be a manager of the project but that's about it. The first thing to do is get a team together and hash out the 'rules' and guidelines. Once the groundwork is laid and everyone understands it it's just a matter of writing code, sharing code, sharing ideas, and learning. Every good software application is planned before the first bit of code is ever written. And usually that plan is updated during the course of development because you cannot possibly think of everything. I'm sure if there is enough interest that we can get our own forum on here....if not, there's always BTS mods or some other place to gather. Say you're using your favorite tool and you wish it could do this or it could do that. If it's open source code then you can make whatever changes you like since you have the source code. It would be neat to be able to add the use of scripting to modules and such. I'm learning Python for the upcoming SH5 release but haven't learned how to integrate it with C++ or C# yet. Last edited by TheDarkWraith; 10-15-09 at 08:44 PM.

10-16-09, 06:27 PM	#7
urfisch Sea Lord Join Date: Mar 2005 Location: Deep down in Germany Posts: 1,969 Downloads: 42 Uploads: 0	i think the idea is great! even i could not help, i am very happy to see racerboy sweepin round here again... __________________

10-17-09, 02:05 PM	#9
skwasjer The Old Man Join Date: Apr 2007 Location: Netherlands Posts: 1,549 Downloads: 26 Uploads: 3	I've left the Dat.dll in S3D on purpose unobfuscated and even included source code for all controllers, just so anyone can (re)use it. I've always encouraged open source for that matter. The problem is the underlying framework is copyright protected so I can't release all of that. I can provide info/insights or examples though, no problem. S3D's file parsers are very flexible and robust by now and so it would be stupid to redesign it. I can help here too. It will all depend on how different or not SH5 will be, and how much the devs provide info on the new storage structures. I've already did some enquiries but sadly no response. But I promised myself if I have to ever use a hexeditor again on SH-data, then you will not see another update to S3D... The amount of time it takes is disproportional, and yet many people just take it for granted. Good luck on the project. Speaking from personal experience, you will need it. You have many challenges ahead of you... __________________ S3D - Silent 3ditor (v0.9.9) - S3D homepage - Open sourced S3D file parser API's - SUBSIM thread SH5 mods: Speech Recognition for SH5 \| Digital UI Clock Tutorials: [TEC] Import/export 3D models to/from game using S3D [TEC] How to work with the model viewer in S3D - VIDEO

10-16-09, 01:41 AM	#4
THE_MASK Ace of the deep . Join Date: Jan 2006 Posts: 9,226 Downloads: 901 Uploads: 73	It would be good if as a modder you could use this program to make more human figures and animations . http://www.gamedev.net/features/revi...?productid=762

11-17-09, 12:38 PM	#10
TheDarkWraith Black Magic Join Date: Jun 2007 Posts: 11,962 Downloads: 147 Uploads: 5	Had to leave country for awhile on assignment but I'm back in the US again. While I was away I decided it would be better to re-design my Apollo application for the upcoming SH5 release to make it more 'modular' and easier to upgrade/modify. Here is the source code for an old version of Apollo in one of it's design stages: http://rapidshare.com/files/30839125...nt_old.7z.html Code is written in C# using Visual Studio 2008. Working on new version currently (in C#) and plan on releasing source code for it once it's in a stable state.