[TEC] The future of S3D / DAT-file format

Saveing with Goblin changes the Type from 10/1000 to 10/-1
Saveing the same file with S3D does not change the types.

Goblin will not save a Type 4/100 Node.
It simpley removes it.
Then all the attached Nodes are now lost but remain in the file.
SH5 places nodes with the Blah-FX.GR2 file.
This is a physical placement Node.
(For lack of a better term)
All Placement Nodes, in all GR2 files, are a 3D placement thing.
They created little boxes and placed them in 3D.
Those objects were Hidden in 3D Max and exported as a GR2 file.
So SH5 'Sees' them.
We add a type 4/100 which 'works'.
Goblin has no idea what the heck they are!
But the Game engine Knows how to use them.
Thank God for that!

[skwasjer]

Originally Posted by Nisgeis

I wouldn't use that as a yard stick. Just because a change doesn't cause problems doesn't mean it's correct syntactically. The game is very forgiving and will accept all sorts of errors without batting an eye lid. But add another error in of the same type and bang, explosion.

I remember Skwas talking of a number of hack jobs from hex editting that on their own didn't cause problems, but did cause damage to the files and problems down the line. So, just because something works, doesn't mean it's right or has the right data structure.

Build on solid foundations and you'l have a stable set of tools. Build on uncertain ground? Who knows what might crop up.

I have always been very conservative in assuming what specific data may mean. This way I can guarantee stable operation of S3D. Especially in the early days, many hex-edited files contained what I believed wrong data, and I was proven right in almost all scenario's. Even today, some mods contain faulty data, and I really regret that these modmakers do not take extra care. Perhaps they do not realise it, perhaps they do. The game does have parsing rules that 'should' be followed, yet has a very relaxed parsing engine, allowing for all sorts of data file errors. This is a problem in itself, because when faulty data is accepted this leads to undefined behavior, or even the data is considered by other modders to be correct when it isn't. One must remember that a tool like S3D can't be as succesfull as it has, if it constantly produced unreliable files with all sorts of errors. So I must have done something right...

In privateers example, he may have a point, we just don't know yet. This is not a problem just yet. I will remain conservative as I always have until I can be conclusive. I have seen more stuff in the game files that were believed to be 'required' (by myself or others) when they weren't. Case in point, the TMAP. Many SH4 models contain 4 uv-maps, where the game actually only uses the first 2. The 3rd and 4th are unused, and in fact add to the file size of the model-files. I have always been skeptical about why 4 uv-maps where included, but could not prove this until 2 years after SH4's release, and I don't think many even know this right now.

The purpose right now of this thread is to gather all this kind of info, and see what we can make of it together.

[skwasjer]

Thanks for the info. It doesn't make it any easier though when the game apparently mixes DAT and GR2 nodes around... yukk!

Originally Posted by privateer

Saveing with Goblin changes the Type from 10/1000 to 10/-1
Saveing the same file with S3D does not change the types.

Does it also do this for the other subtypes (ie.: 10/0 to 10/6?)

[Nisgeis]

Originally Posted by privateer

What part of "no issues as yet identified" is not clear to you or others?

Hey, I just posted a cautionary tale. Your reaction is a little extreme and quite agressive... What part of 'blah' don't you understand? Oh please.

Originally Posted by privateer

I am looking for solutions.

When trying to diffuse a bomb, advice of 'Don't kick the bomb' is just as valid as other advice.

Originally Posted by privateer

Not stateing it's the only way.

I didn't say you were, I just say hey, be careful about assuming that's gospel just because it worked.

Originally Posted by privateer

The whole thing is going to be a path of discovery.

Yes, with lots of people pitching in with ideas and comments. So probably the worst thing you could do is to have a go at someone who is trying to help no?

Originally Posted by privateer

We will find what's right and wrong as we go.
SH3 and SH4 went through this.
We try what worked with SH3/SH4 and if it don't work?
We find out why and move forwards.

Yes, hence my 'hex editting causes problems cautionary post. Hex editting has caused problems in the past where people didn't edit the files to the correct standards.

Originally Posted by privateer

Just because We got Goblin does not make it the end all be all Tool.

I didn't say it was. Obviously it isn't the be all and end all tool as it doesn't do half the jobs.

Originally Posted by privateer

S3D was not nor is the end all be all Tool.

It's the best tool so far to emerge from the community bar none.

Originally Posted by privateer

A Hex Editor is also not the end all be all Tool.

Agreed.

Originally Posted by privateer

The finest Tools out there?
Knowledge , willingness to attempt something,
and refuseing to believe it can't be done or figured out.

How about you show willingness to work with the community without posting snotty posts like you just did? I always defended you and I thought your relapse in the SH4 ATO thread was a one off, but now I'm not so sure. Either you really do want to build a better SH5 modding toolset, or it's just a sham.

Originally Posted by privateer

SH5 is a new Game.

Yes, so let's hope for a new spirit of co-operation and not the same old BS yeah?

Originally Posted by privateer

You nor I know for a stone cold fact what is what as a whole.
I use what I learned in the best way I can to assist moveing forward.

I'm really not pretending to know 'what is what as a whole' whatever that means. I'm just saying don't take some hex editted gubbins as a solid foundation just because there have been no issues with it.

Originally Posted by privateer

Will we all make mistakes?
You better believe it!
We will FUBAR everything we can until we have all the answers!
If stacking on unknown controllers leads to a CTD?
I want to know about it and why!
Then look to adjust.
Theories at this point?
Are just theories.

I have a theory that if someone posts a post trying to help, then it would be good if someone else didn't post a big long winded post about nothing in particular as that really will put off other people from helping, in case they got the same quote and speech response. Can't help in a way that Privateer wants? Then you just can't help and you must suffer a long speech? Is that the message you want to send? In any case, a long winded response is required for a long winded post.

Now Privateer, look what you've done. You've turned a thread about betterment of the community into something else. Why would you do that when I was just trying to help? Why would be so so snippy? Why? Why?

What part of 'Let's work together' is not clear to you or others?

I have to say skwasjer is alot more Silver Talking then me.


We have his attention Guys.
Take advantage of this as long as it lasts.
I've bantered back and forth with skwasjer for a long time now.

My main challange to him in an off hand way was to get me to
quit useing a hex editor.

Well he's come DAMNED close!


If I can convince him to add one or two more things?
I'll retire the Basic Hex editors and only use S3D.

[TheDarkWraith]

Originally Posted by privateer

My main challange to him in an off hand way was to get me to
quit useing a hex editor.

Well he's come DAMNED close!


If I can convince him to add one or two more things?
I'll retire the Basic Hex editors and only use S3D.

You can't retire the best tool that can do anything you want it to do if you have the knowledge I refuse to use any tool as it is nothing more than that - a tool that is only as smart as the person who made it (not putting down anyone's programming skills) and does what that person programmed it to do. A tool also stifles your creativity as it defines the rules and what's acceptable and not - hex editor doesn't care

[skwasjer]

Oh my...

TDW, some of your mods are exactly an example of what I was talking about. At least privateer in the past embraced some of my remarks regarding file errors in the GWX mod about 2 years ago and corrected them, and is willing to accept that some rules are to be followed (if only for the sake of community sanity). You can't just throw out any kind of mod that supposedly works, while it contains errors that can be avoided. You can do whatever you wish with S3D, I am not enforcing S3D on you or him, nor do I believe he will ever stop using one (nor do I see a reason why you or him should), but you could at least be more appreciative of what S3D has achieved in the community: more people making more advanced mods than 5 guys with a hex editor could have done without S3D's existance. And you could take that to your advantage by using it for many tasks it 'does' better than a hexeditor...

Now can we at least drop this discussion? I did not start it to start bashing at S3D, hexeditors or other tools. This is so pointless...

I don't think anyone believes removeing Hex Editors completely is the goal.
And I mean Hex Editors as in Guys who know Hex and work in it.
It's a fact Modders have done great things with S3D with no knowledge of Hexing things as you and I do.
You and I also know those same Modders would not learn Hex as you and I know it.
It's a gift and a curse in ways.
I accept the fact that S3D can do many things just as I do them.
I also point out what it can not do.
In an attempt to improve.

You once took great care that things worked.
And I can remember you teaching me many times in the past.

You, skwasjer, me, and who ever else stands up and provides input
that advances things?

That's the whole deal.

Some of us may be faster, better, and more knowledgeable then others.
To me that means I owe them.
Not that they owe me.

[skwasjer]

I must point out, many oldskoolers but also many newcomers have had significant input in S3D's long running development (just look at the changelog). So S3D is not a product of me alone, the developer. Alot of community input I used to improve the tool with. This is what I am asking now again, only this time I ask it ahead of time, not during or afterwards...

No point in putting your head in the sand. Instead, step up to the plate and make a difference.

[SteelViking]

Once again, I don't have much in the way of technical help, but I can be relied upon for suggested features. I have been working with S3D a pretty good deal lately to see what it is capable of in regard to SH5, and the 3rd post of this thread contains my preliminary "wish list". To reiterate one of the wishes I made, I would love if S3D could be made to open GR2 files just to give us the ability to see the 3d models to make editing things a bit easier.

One more thing I can think of that would be a huge help is: would it be possible for S3d to be able to add way points for the crew. I know that the current waypoints are untouchable, as they are in a GR2 file, but would it be possible to add them to another type of controller file. Others have run into walls before when it comes to this issue. Heretic in particular noted that he was very limited in what the crew could be made to do because there were simply not enough waypoints available to script more behaviors into the game.

[urfisch]

Nice, to see some new progress on that topic, skwasjer. Here are the templates i changed for the fx effects in march.

http://rapidshare.com/files/363694792/Controllers.zip

I experienced the same problem, as stated by privateer. controller heads are lost, when opening the changed dat in goblin. This is where i got stuck. But changing the most controllers for particles is possible.

My Post:

http://www.subsim.com/radioroom/showthread.php?t=165022

[skwasjer]

Thanks urfisch.

[iambecomelife]

Originally Posted by skwasjer

Since I've indicated I will (likely) release an updated to S3D, and some people are trying to make S3D work (read: mod) with the new data built on top of SH4's DAT-file format I will make this new thread here in the SH5 section. This thread is to ask anyone of you to publish any info you have found regarding the file format, any customization you made to S3D's controller templates, etc. and perhaps even a wishlist for the next installment of S3D.

Provided enough info is gathered, and alot of prework is done, this saves me from having to work on S3D all by myself, something I can no longer do like I have in the past. I will take any info you have, and consider any question/tip. Remember that the more you help out, the more likely development on S3D will be continued, as a (partial) replacement for the Goblin editor, a tool that lacks in many departments.

At least privateer has done quite a few modifications already (here and there with my help), and I hope more of you can help out. Obviously, you will receive credit for your efforts, as many users and other modders will benefit from your support.

If you have (technical) questions or need pointers related to S3D or hex, you can also ask them here. Me or privateer or someone else may be able to answer.

Perhaps privateer can post the links to all his S3D-mods, so I can copy them in the first post.

Afraid I can't add very much to the community's technical knowledge, but I do have a few requests for any future version of S3d (or its replacement):

-Ability to see weapons attached to nodes (presently, ships appear without turrets etc)

-Ability to preview particle effects

-Most importantly, enhanced 3d model importation. Namely, compatibility with non-triangulated .obj files. Currently, if I forget to triangulate a single item the whole ship or plane fails to import.

[skwasjer]

I understand how these functions would help, but I'm not looking to work another year on S3D. I am primarily focussing on adding support for missing pieces of data in SH5. They are noted, but don't count on it.

Here's what I have so far.
Packed up as I use them with JSGME.
http://www.mediafire.com/?ma53n13qrgq2mni

New today is the obj_Turret and unit_Ship
(I just did a Vector3 on the ship_color for now)