Shall we talk about file sizes in mods?

[Jeff-Groves]

Originally Posted by Hitman

Jeff you have raised one of the most important points ever here. So many years we have been following the inertia and ignoring how the snowball became ever bigger while rolling. There is no doubt the 3D part is a huge hit for our processors - having gone to much more modern PCs since 2005 has masked it but still on old game engines like SH3 becomes noticeable. Yes, shared resources in the 3D part are key to solve many problems but we have a formidable number of scattered units around, some in super mods, others in single unit mods/addons

Therefore the best way to do an optimization that works for everyone would probably be to list the units and proceed in a collaborative effort to slowly make our way through it. I guess that IABL merchant fleet mod is one of the most spread around mods, but I have no idea if he already did that optimization.

I believe a program written to scan every dat file and remove the useless stuff would be the way to go. Then anyone could download it, run it, and carry on without the hassle of reinstalling and so forth. No MFM has not been optimized. It has 125 Units with the stuff.

[Jeff-Groves]

For converting SH3 units to SH4? There is a program that will do all the hard work for you.
It converts the unit completely! only thing it does not do is create the AO mappings.
All dynamic shadow stuff is ripped from the file along with the 3D models,
Unifiedrender is installed, textures are converted to dds, val and other files fixed also.

So it NEARLY readies SH3 units for SH5 also!

[Jeff-Groves]

I just scanned the complete data folder of GWX.
I found 244 files with dynamicshadow.


My scan lists the names and locations of every file found also.
I'm adding to the script to give me exact locations in each file.
From there I should be able to build up to removing things.

[HanSolo78]

Your programme would do a hell of work!

If you have a look into the WAC full interior you will notice that there is also a lot to optimize.
http://www.mediafire.com/file/jlrdsd...terior.7z/file

I started with remeshing 3d meshes and reducing triangles, etc.
Also.. very time consuming... I only got 1 interior done (not published yet).

[Jeff-Groves]

WAC scan of data folder shows 236 files total. So yes. Could be cut way down.

[Anvar1061]

Jeff, give instructions and together we can work on perspective in my opinion project WAC v5.02.

[areo16]

I think this is a damn good idea. Reason for many crashes too is not just id conflicts but hitting the 4gb limit for the 32 bit application which sh3 is (with 4gb patch).

[areo16]

Originally Posted by Jeff-Groves

I believe a program written to scan every dat file and remove the useless stuff would be the way to go. Then anyone could download it, run it, and carry on without the hassle of reinstalling and so forth. No MFM has not been optimized. It has 125 Units with the stuff.

We have S3D to open the .dat file and show its objects and relationships but I don't think we have a .dat decompiler that can output some sort of file format that is readable to such a script.

[Jeff-Groves]

Originally Posted by areo16

We have S3D to open the .dat file and show its objects and relationships but I don't think we have a .dat decompiler that can output some sort of file format that is readable to such a script.

Don't need to decompile the dats.
I can read them in Hex as many of the old modders did from the start.
Some of us still know the format of the dats.
Here's a little sample:


SH III DAT FILE FORMAT
Skip the first 20 bytes they seem to be always the same.
Next 4 bytes are a long with the header string lenght.
Next is the author name and the "using Kashmir" stuff, total lenght as read above.
Next, file is made of "chunks". All chunks are like this:
ChunkType 4 bytes
Chunk Subtype/Version 4 bytes
chunk_size long (4 bytes)
Data (chunk_size bytes)

Chunk types and internal structure of each one:
type 0: EMPTY chunk
it's just 12 "00" bytes, so type, subtype and size are 0
type 1: 3D chunk
after the Chunk type, subtype/ver and size, format of the data part is like follows:
chunk_id: 8 bytes //this is used to reference this part inside the file. References usually found inside NODE (type 4) chunks


010 can be scripted to do what ever one wants to do to the dats in that case.
S3D is a fine Tool! Many of us old modders contributed information during it's development.
But it has it's limits. Working with a great Hex editor excides what S3D can do.

[Jeff-Groves]

4096 words of random access memory. That wouldn't have been the 704's would it?

"I suggest that there are many projects which could use modder effort more effectively than cleaning up old inefficient models which do run adequately on current machines."

adequately

There's the word I hate.
The program runs adequately as is but is in-efficient and can load faster, reduce possible ID conflicts, and allow the complete interiors to work better.
(They have been done for several Mods)

[ETR3(SS)]

Originally Posted by Jeff-Groves

adequately

There's the word I hate.

Better is the enemy of 'good enough.'

[Jeff-Groves]

Originally Posted by ETR3(SS)

Better is the enemy of 'good enough.'

Not sure which interpretation of this you intend?


We've been running SH3 the 'good enough' way since 2005.
In my 40 year trade in construction? I learned 'good enough' was just a way to justify sub par work.
I do not accept 'good enough' when training our employees or doing the work we are hired to do.

[ETR3(SS)]

Sorry for leaving that ambiguous; it was an agreement with your statement on adequately.

I've run into more than my fair share of good enough and it drives me nuts, particularly when I'm the one that has to make it better.