SUBSIM Radio Room Forums


SUBSIM: The Web's #1 BBS for all submarine and naval simulations since 1997

Go Back   SUBSIM Radio Room Forums > Silent Hunter 3 - 4 - 5 > SHIII Mods Workshop
Forget password? Reset here

Reply
 
Thread Tools Display Modes
Old 11-27-2018, 12:41 AM   #31
areo16
Weps
 
Join Date: Oct 2013
Posts: 363
Downloads: 58
Uploads: 0
Default

I think this is a damn good idea. Reason for many crashes too is not just id conflicts but hitting the 4gb limit for the 32 bit application which sh3 is (with 4gb patch).
areo16 is offline   Reply With Quote
Old 11-27-2018, 12:50 AM   #32
areo16
Weps
 
Join Date: Oct 2013
Posts: 363
Downloads: 58
Uploads: 0
Default

Quote:
Originally Posted by Jeff-Groves View Post
I believe a program written to scan every dat file and remove the useless stuff would be the way to go. Then anyone could download it, run it, and carry on without the hassle of reinstalling and so forth. No MFM has not been optimized. It has 125 Units with the stuff.

We have S3D to open the .dat file and show its objects and relationships but I don't think we have a .dat decompiler that can output some sort of file format that is readable to such a script.
areo16 is offline   Reply With Quote
Old 11-27-2018, 03:23 AM   #33
JapLance
Sonar Guy
 
Join Date: Feb 2006
Location: Palma, Spain
Posts: 379
Downloads: 182
Uploads: 0
Default

Quote:
Originally Posted by Jeff-Groves View Post
Water Reflections in most cases is the same.
No need for a separate 3D mesh.
It's been a long time since I tweaked with SH3/SH4 files, but I have to disagree here.

The water reflections don't need the full poly model to be displayed. A low-poly model can be safely used (not actually noticeable in the game) and lessen the charge on the GPU.

And I eagerly await for that program to convert SH3 models into SH4. I remember it was a painful, time consuming job to do.
__________________
Greetings from Spain
JapLance's ships for SH4
JapLance is offline   Reply With Quote
Old 11-27-2018, 04:22 AM   #34
bigboywooly
Rear Admiral
 
Join Date: Apr 2006
Location: Swindon, England
Posts: 10,142
Downloads: 30
Uploads: 0
Default

Quote:
Originally Posted by Jeff-Groves View Post
BBW,
Take a look at the NBB_Nevada in the SH4 sea folder.
(I can send it to you if you don't have SH4)
You might recall I tested this just after SH4 came out as We were putting the BuV_Dock1 and BuV_Dock2 into GWX.
Hmmm, have SH4. Will take a look
__________________


My mediafire page http://www.mediafire.com/?11eoq19bq9r41
bigboywooly is offline   Reply With Quote
Old 11-27-2018, 09:10 AM   #35
BigWalleye
Sea Lord
 
Join Date: Jul 2012
Location: Beyond Black River - in Pict country, you betcha!
Posts: 1,985
Downloads: 465
Uploads: 0


Default

Some thoughts from an old programmer:

Once, a very, very long time ago, I worked on the operating system for a computer which had exactly 4096 words of random access memory. That's not a misprint! Not K. Not M. Four thousand ninety-six 18-bit words. (A beer for anyone who can correctly identify that machine!) Every instruction had to be carefully chosen. Every status bit had to be considered for maximum added functionality. For the size of the code, it required a huge investment in programmer effort.

Later, I worked on a real-time virtual OS which, for sound physical reasons, was based on magneitc drum storage. The latency time of the slowly-turning drum meant that data and executable storage had to be carefully optimized to be under the heads when needed. There was software to help, but any significant change to data structures meant that the entire drum had to be remapped. Again, massive amounts of effort to overcome hardware limitations.

Throughout the history of computing, huge amounts of human programmer effort have been used to compensate for the limitations of the existing hardware. I'm sure Ada Lovelace had the same problems.

When SH3 was developed, the 3D models taxed the generally available hardware to the point where load times were barely tolerable. As SH3 grew through mods, much more extensive modelling made use of improving machine capabilities, and load times stayed barely tolerable. So we are today.

I am sure it would be possible, through large amounts of modder effort, to optimize the 3D models with standard elements. This would likely make a significant improvement in load times, especially for older hardware. (Would it also add significant render times and stress older CPUs and GPUs?) But is this the most effective use of the time and talents of a small modding community? The Polish Engineer sez: "If it ain't broke, don't try to fix it."

There are still many possible improvements to SH3 waiting to be done. In reverse enginnering, hsie did wonderful things, but much more is possible. (Compare to what TDW has done for SH5.) While the skill set for 3D modelling is not the same, still I suspect there are many useful projects to be done. (How about a complete interior?) I suggest that there are many projects which could use modder effort more effectively than cleaning up old inefficient models which do run adequately on current machines.

Truth in advertising: I am not an active modder, except for personal consumption. FWIW, this is just the opinon of a long-time software developer. YMMV.

And for all those who labor to make this old horse better and better every year, THANK YOU. Much love, and I will be grateful to use whatever you care to contribute to the community. God bless!
BigWalleye is online   Reply With Quote
Old 11-27-2018, 02:03 PM   #36
Jeff-Groves
Village Idiot
 
Jeff-Groves's Avatar
 
Join Date: Sep 2014
Location: Nevada, Ohio
Posts: 2,216
Downloads: 42
Uploads: 0


Default

Quote:
Originally Posted by areo16 View Post
We have S3D to open the .dat file and show its objects and relationships but I don't think we have a .dat decompiler that can output some sort of file format that is readable to such a script.
Don't need to decompile the dats.
I can read them in Hex as many of the old modders did from the start.
Some of us still know the format of the dats.
Here's a little sample:


SH III DAT FILE FORMAT
Skip the first 20 bytes they seem to be always the same.
Next 4 bytes are a long with the header string lenght.
Next is the author name and the "using Kashmir" stuff, total lenght as read above.
Next, file is made of "chunks". All chunks are like this:
ChunkType 4 bytes
Chunk Subtype/Version 4 bytes
chunk_size long (4 bytes)
Data (chunk_size bytes)

Chunk types and internal structure of each one:
type 0: EMPTY chunk
it's just 12 "00" bytes, so type, subtype and size are 0
type 1: 3D chunk
after the Chunk type, subtype/ver and size, format of the data part is like follows:
chunk_id: 8 bytes //this is used to reference this part inside the file. References usually found inside NODE (type 4) chunks


010 can be scripted to do what ever one wants to do to the dats in that case.
S3D is a fine Tool! Many of us old modders contributed information during it's development.
But it has it's limits. Working with a great Hex editor excides what S3D can do.
__________________
I have not failed. Ive just found 10,000 ways that wont work.

Last edited by Jeff-Groves; 11-27-2018 at 02:41 PM.
Jeff-Groves is offline   Reply With Quote
Old 11-27-2018, 02:10 PM   #37
Jeff-Groves
Village Idiot
 
Jeff-Groves's Avatar
 
Join Date: Sep 2014
Location: Nevada, Ohio
Posts: 2,216
Downloads: 42
Uploads: 0


Default

4096 words of random access memory. That wouldn't have been the 704's would it?

"I suggest that there are many projects which could use modder effort more effectively than cleaning up old inefficient models which do run adequately on current machines."

adequately

There's the word I hate.
The program runs adequately as is but is in-efficient and can load faster, reduce possible ID conflicts, and allow the complete interiors to work better.
(They have been done for several Mods)
__________________
I have not failed. Ive just found 10,000 ways that wont work.

Last edited by Jeff-Groves; 11-27-2018 at 02:32 PM.
Jeff-Groves is offline   Reply With Quote
Old 11-27-2018, 03:26 PM   #38
Jeff-Groves
Village Idiot
 
Jeff-Groves's Avatar
 
Join Date: Sep 2014
Location: Nevada, Ohio
Posts: 2,216
Downloads: 42
Uploads: 0


Default

Quote:
Originally Posted by JapLance View Post
And I eagerly await for that program to convert SH3 models into SH4. I remember it was a painful, time consuming job to do.

Check your PM's.

Your review of the program would be great!
__________________
I have not failed. Ive just found 10,000 ways that wont work.
Jeff-Groves is offline   Reply With Quote
Old 11-27-2018, 03:37 PM   #39
ETR3(SS)
Ocean Warrior
 
Join Date: Dec 2007
Location: Too far from the sea...
Posts: 2,811
Downloads: 165
Uploads: 15


Default

Quote:
Originally Posted by Jeff-Groves View Post
adequately

There's the word I hate.
Better is the enemy of 'good enough.'
__________________


USS Kentucky SSBN 737 (G)
Comms Div 2003-2006
Qualified 19 November 03

Yes I was really on a submarine.
ETR3(SS) is offline   Reply With Quote
Old 11-27-2018, 03:52 PM   #40
Jeff-Groves
Village Idiot
 
Jeff-Groves's Avatar
 
Join Date: Sep 2014
Location: Nevada, Ohio
Posts: 2,216
Downloads: 42
Uploads: 0


Default

Quote:
Originally Posted by ETR3(SS) View Post
Better is the enemy of 'good enough.'
Not sure which interpretation of this you intend?


We've been running SH3 the 'good enough' way since 2005.
In my 40 year trade in construction? I learned 'good enough' was just a way to justify sub par work.
I do not accept 'good enough' when training our employees or doing the work we are hired to do.
__________________
I have not failed. Ive just found 10,000 ways that wont work.
Jeff-Groves is offline   Reply With Quote
Old 11-27-2018, 04:23 PM   #41
ETR3(SS)
Ocean Warrior
 
Join Date: Dec 2007
Location: Too far from the sea...
Posts: 2,811
Downloads: 165
Uploads: 15


Default

Sorry for leaving that ambiguous; it was an agreement with your statement on adequately.

I've run into more than my fair share of good enough and it drives me nuts, particularly when I'm the one that has to make it better.
__________________


USS Kentucky SSBN 737 (G)
Comms Div 2003-2006
Qualified 19 November 03

Yes I was really on a submarine.
ETR3(SS) is offline   Reply With Quote
Old 11-27-2018, 04:30 PM   #42
Jeff-Groves
Village Idiot
 
Jeff-Groves's Avatar
 
Join Date: Sep 2014
Location: Nevada, Ohio
Posts: 2,216
Downloads: 42
Uploads: 0


Default

Quote:
Originally Posted by ETR3(SS) View Post
I've run into more than my fair share of good enough and it drives me nuts, particularly when I'm the one that has to make it better.


Kind of thought so.
And I'm the one that gets sent 500+ miles from home to fix 'Good enough'.
__________________
I have not failed. Ive just found 10,000 ways that wont work.
Jeff-Groves is offline   Reply With Quote
Old 11-27-2018, 06:41 PM   #43
BigWalleye
Sea Lord
 
Join Date: Jul 2012
Location: Beyond Black River - in Pict country, you betcha!
Posts: 1,985
Downloads: 465
Uploads: 0


Default

On the day when we have all the time in the world and all the manpower in the world, we will not have to prioritize anything, and everything that needs to be done will get done. Until then, I personally believe adding new capabilities to SH3 is a better use of scarce modder time than cleaning up existing mods that work adequately. But that's just my opinion. As I pointed out, I am not modding, and what project any modder chooses to work on is a strictly up to them. I just wanted to lobby for considering an alternative.

My interpretation of "Better is the enemy of good enough." is the one endorsed by Winston Churchill, who had considerable experience in effectively prioritizing limited resources. (AFAIK, it was Churchill who coined that aphorism, or at least popularized it.)

And no, it was not a 704. The 704 used two different instruction formats, but each one was 36 bits long. And the addressing was 36-bit oriented. A hint: It was a mini-computer and the bulk storage device was called DECtape.
BigWalleye is online   Reply With Quote
Old 11-27-2018, 09:20 PM   #44
propbeanie
CTD - it's not just a job
 
propbeanie's Avatar
 
Join Date: May 2016
Location: The Crossroads of Mid-America
Posts: 4,571
Downloads: 246
Uploads: 0


Default

All he's talking about doing is removing un-used "code", if you will (bloatware), so the data will not only fit on the tape drive, but also be in a nice orderly manner...
__________________

"...and bollocks to the naysayer/s" - Jimbuna
propbeanie is offline   Reply With Quote
Old 11-28-2018, 01:31 AM   #45
areo16
Weps
 
Join Date: Oct 2013
Posts: 363
Downloads: 58
Uploads: 0
Default

Quote:
Originally Posted by Jeff-Groves View Post
Re-using 3D meshes in a file.
I'll provide the example.
http://www.mediafire.com/file/4afldw...sell2.dat/file

Look at this dat with S3D. Then look at how the actual 3D meshes were done. Notice there are very few actual meshes used to complete the Sea Forts. The Game engine renders the needed parts as we placed them. No need for extra meshes!
At 2+ MB as is? Doing everything as separate meshes would place it at over 9 or 10 MB!
(Yes. They suffer the dreaded Dynamic shadow stuff! I know better now.)

Thanks for the offsets for the .dat file in hex very helpful.


So I understand the optimized approach you are talking about here, you are still copying the same model (obj) over and over again in S3D?


Isn't it possible to just reference the same model in S3D and just go about it that way, wouldn't that save space?


I guess, from a hex point of view, what edits would we want to make to these files to make them more optimized. Looks like looking for duplicate data chunks for models (type 1) is step 1, and then possibly deleting the duplicates and re-reference the deleted duplicate data chunks with the original model reference id?


@EDIT


Yes in S3D it is possible to reference the same model in multiple nodes and this works just fine. It also decreases file space.


Is it possible to reference a model ID that is in another .dat file?

Last edited by areo16; 11-28-2018 at 02:50 AM.
areo16 is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -5. The time now is 07:27 AM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
Copyright 1995- 2019 Subsim