SweetScape 010 Editor

[Jeff-Groves]

I asked the Moderators here at SubSim where would be the best place to start this thread. After suggestions and such I feel the SH4 Workshop is the best place to start since I convinced several people to give it a try.

010 is a hex editor with almost magical powers! I'd HIGHLY suggest one check out their home page for some details as it would take pages here to just give you the basics.

http://sweetscape.com/

It is a pay for program after 30 days but WELL worth the low cost! I can do anything to the files that all other programs can do AND much more!

If you can imagine it and can or are willing to learn a bit of C programing?
There is no limit to what you can do.

I plan to release some scripts and templates here that work with 010.
Those will show you and allow you to do things other programs can not do easily for the SH series of Games, do the things that take multiple programs, automate things to cut the time needed to do things.

Another great thing is the creators of 010 listen to suggestions for improvements to 010 like no other software devs I've ever had dealings with!

The list of 010 users here at SubSim is amazing when you look at what they have done for the Games! I'll hope they list themselves and if they are no longer here at SUbSim? I'll add them.

To be continued........
:up:

[Jeff-Groves]

The sharing of scripts and templates for 010 is easy.
Below is a script that you can use to search and save a list of output.
Just copy and paste it to a text file. Then give it the extension of 1sc instead of txt.
It's a simple script that you can actually do with the user interface but does save some time.

//------------------------------------------------
//--- 010 Editor Script File
// FotRSU Search Script
//
//------------------------------------------------

int i;
char W[40], X[20], Y[20];


X = InputString("File Search", "Enter file name", "*.dat"); // files to search for
Y = InputString("Text Search", "Enter text", ""); // This is the text to find

Printf( "You searched for %s\n", Y );
TFindInFilesResults r = FindInFiles( Y, InputDirectory( "Select Folder to search", "C:\\" ), X );

Printf( "I found %d files.\n", r.count );

for( i = 0; i < r.count; i++ )
{
Printf( "%s\n", r.file[i].filename );
}
W = InputString("Save File", "Name your save file", "c:\\temp\\");// You can change the path when saving.
OutputPaneSave( W );

[Jeff-Groves]

010 allowed me to open the Bunker in SH5 so you can walk areas not allowed in the original version. I don't believe it's ever been copied.

http://www.mediafire.com/file/141175d8yg5d7vy/Bunker_Newest.wmv/file

[Jeff-Groves]

Now say you use the script I posted above and find 250 files you need to check.
Sounds pretty daunting don't it?
What IF you could run a script that opened each file in S3D as you clicked on them? And then marked the list so you could come back later to continue your work?
S3D would close and you could just click the next link in the list.
Talk about working on multiple mods time saving!

I wish 010 had been around for GWX!!

By the way? I don't use anything other then 010 to set large address aware.

[propbeanie]

I've got a Toobyue account Jeff. Do you want that video online? :salute:

[s7rikeback]

I asked the Moderators here at SubSim where would be the best place to start this thread. After suggestions and such I feel the SH4 Workshop is the best place to start since I convinced several people to give it a try.

010 is a hex editor with almost magical powers! I'd HIGHLY suggest one check out their home page for some details as it would take pages here to just give you the basics.

http://sweetscape.com/

It is a pay for program after 30 days but WELL worth the low cost! I can do anything to the files that all other programs can do AND much more!

If you can imagine it and can or are willing to learn a bit of C programing?
There is no limit to what you can do.

I plan to release some scripts and templates here that work with 010.
Those will show you and allow you to do things other programs can not do easily for the SH series of Games, do the things that take multiple programs, automate things to cut the time needed to do things.

Another great thing is the creators of 010 listen to suggestions for improvements to 010 like no other software devs I've ever had dealings with!

The list of 010 users here at SubSim is amazing when you look at what they have done for the Games! I'll hope they list themselves and if they are no longer here at SUbSim? I'll add them.

To be continued........
:up:
May I check in, do you have a spare room ?

[Jeff-Groves]

Thanks for checking in Mates!
:salute:
You Guys can attest to what 010 can do for us Modders I'm sure.

I'm going to blast you all with some really amazing scripts to edit animations in SH4 here soon!

There is no other software that will give you that power!
S3D can not even open them let alone edit them.

[s7rikeback]

Is there any way someone could scan the texture folders within a mod for a date range? so it would flag up anything in a certain date range... or even better could it show the file size of the dds / tga file and the date range ? as in 128x128 or 512x512.

[Jeff-Groves]

Most excellent question!! I'll start with scanning for texture size.

Say we want to find a TGA of a certain size. That info IS in the TGA file so we would scan all files to find the image specifications which are in every tga file. Even in a dat file in SH Games.
We need to understand that format to start the search.
If you have 010 open the iceburg3.dds in your texture folder. (Ya. I'm switching you up to dds but it's the same)
You'll get a popup to install a template. Go ahead and install that. Now you will see how that file is built. Click on the struct DDS_HEADER and you will see the size of that dds file. Unit height is 00 02 00 00 followed by 00 04 00 00
which is 512x 1024

We have the start of a search. Lets be sure we only look for a dds file.

I'll just do a user interface search at this point. Click on search, find in files
put 00 02 00 00 00 04 00 00 in the search box, browse to the SH4 texture folder, and put *.dds in the file types area.

Now this can be scripted also! We can script it so you get a box that will take 512 and convert it to the hex numbers!
Holy Hanna! You don't even need to know hex if a script is written!

[Jeff-Groves]

The time date stamp is a bit trickier. But can still be coded to find.
For instance. The SH3.exe is dated 3E 9B AE 42
Or should I say Tuesday, ‎June ‎14, ‎2005, ‏‎8:54:24 AM
Given that info there must be a date stamp in the other files We have missed or not understood!

[Jimbuna]

Stickied :salute:

[oskar123]

I looked through their site but I have to say this looks complicated and I don't really understand it. I'm willing to learn of course but I don't really know where to start.

[propbeanie]

It is not something that can be "mastered", or even used efficiently in just a few mouse clicks, unless you are already familiar with a similar proggie. At its basic level, it is a text editor, and can be used as an editor for programming C or Basic (and hex) or a myriad other programming languages, including scripting, or used just with plain text files. It can use templates to open files so that the user can more easily "see" a files structure, and it can use scripts to more easily utilize said file structure in a semi-automatic way.

010 is a tool, such as an electric saw with changeable blades. You can con-struct something, or you can de-struct something. I've been attempting to use it to pull certain data from certain files for studying and comparing. Thus far though, I'm having to take baby steps, even though I used to do C programming many many moons ago. I am better at getting "Invalid data type" or other error messages, than I am at getting usable results. It takes a while, and a lot of reading of the 010 documentation - and they have a lot of it. Most of us will probably never reach the level of understanding of the editor that Jeff-Groves has, but we can try, and have fun along the way... :salute:

[Jeff-Groves]

s7rikerback,

On the search for a dds file of a certain size? The opensearch.1sc can do it as is. Now I will refine it a bit to be more specific but it does work.

Put *.dds as the files to search and the size would be in what to look for.

Say you want to find dds files of 512
It will pull all dds files with 512 in the file size.
BUT! It will find the 512x1024 and such files. Thus the need to refine the search code a bit.

On the date of files search. 010 could be scripted to do this.
BUT!
Since that type info is stored in the hidden areas of a hard drive? It may be better to use other programs to do that for now.
Reason I say that is IF you make a mistake in that area you can easily trash a HD!
What I'm working with now is to copy FotrsU to a removable drive that has been formatted so it's clean. Nothing else is on it.
I can open and scan the dates of files in the hidden area of the master file table. This gives me the names and all the info we need.
As an example, put a usb stick in your computer. Open 010 and select open drive.
You'll get a prompt to install a template file. Do that then browser what pops up.
Best way to do that is format a usb stick then copy one file to it. Makes it easier to see and understand.
I'd suggest checking the read only box at first!

Now, my system has an SSD drive. Not sure why but I can not open that drive. Thus the copy to a removable drive.
What I'm looking at doing is to just copy the info (Names and dates stuff) we want to a file that you can then sort or search.

[Jeff-Groves]

Saw a post about black units in SH5.
Did a quick look at files and saw a problem with the Units mentioned.
Pointers in the Rosters file to a file that does not exist.
Offered a 010 help on how to find any Units that do not have Normals UV mappings.
http://www.subsim.com/radioroom/showpost.php?p=2585208&postcount=37

Granted many Units from SH3 have TMAPs? most don't have Normals and most custom or modified have NO TMAPS.
But it will cut HOURS checking files!

This also applies to SH4!

[Jeff-Groves]

//------------------------------------------------
//--- 010 Editor Script File
// RAW file reader .001
// Reads the DetailMask RAW files in SH5 and outputs the height values
// Outputs 101 lines of 101 floats
//------------------------------------------------

int i, l, t, s=101;
char W[40];


float r;
for( i = 0; i < s; i++)
{
l=101;
Printf( "Line %d\n", i);
{
for( t = 0; t < l; t++)
{
r = ReadFloat();
Printf( "%f ", r );
FSkip(4);
}
}
}
W = InputString("Save File", "Name your save file", "c:\\temp\\");// You can change the path when saving.
OutputPaneSave( W );

[Jeff-Groves]

The script above is undergoing changes to try something gap and I are working on.
:03:

[Jeff-Groves]

Here are 2 scripts I did for 010
Tangent and Cosine

010 does not use a perfect C code so I had to adjust.
Cosine corrected..............

#define PI 3.14159265

double param, result;
param = 60.0;
result = Cos ( param );// Correct for SweetScape 010
Printf ("The cosine of %f degrees is %f.\n", param, result );


Tangent corrected..........


#define PI 3.14159265

double param, result;
param = 60.0;
result = Tan ( param );// Correct for SweetScape 010
Printf ("The Tangent of %f degrees is %f.\n", param, result );

[Jeff-Groves]

Let's do something handy with 010 and S3D!
Open 010 and go to Tools\Options\Programs
Select the New button and then name it S3D
browse to the S3D install and select the S3D.exe
For arguments select filename.

OK it and then open a dat file in 010
Select Tools again and you will see S3D listed.
Go ahead! Click on it!
:D

[propbeanie]

ooohh - pretty toys... cool. I still can't decipher hex anymore though... :har:

that little menu now works for the sim, dsd, etc files also.

Thanks Jeff! :salute:

PS: ya know what? This is nice with a dual-monitor setup - or if I only had a 'modern' widescreen thingie...

Also, as before, if you want to open more than one file associated with S3D, you do have to go through the S3D interface of "File" "Open", if you are after having two different dat files open at the same time...

[Jeff-Groves]

widescreen thingie...
Isn't that a projector and a white sheet hung on the wall?
:hmmm:

[propbeanie]

:har: The boys got a "projector" that could do that from their computers a few years ago. It does distort things though, if not perpendicular to the surface. But don't mess with the sheet - just paint the wall white. Maybe even go outside and use the garage door... :har: :salute:

Say, uh, you can build a script to find and potentially validate the links between the different S3D files much easier with this little technique, eh?... :salute:

[Jeff-Groves]

You could write a script that you put in, say an ID, that would scan all files for that ID and give you a list.
There's a program that does that already.
http://www.mediafire.com/file/ci2dd7cbjj34aoh/IDDuplicateFinder.7z/file
Problem with that program is not being able to actually narrow things down.

[propbeanie]

Yeah, but I was thinking of a different approach. Then I remembered that no two dat or sim or whatever files are laid-out the same way, the nodes in the same order or names... so my idea wouldn't work anyway. Might work for the first set of files, but bomb on the next... :salute:

(btw, that is a useful utililty - thank you for that)

[Jeff-Groves]

Yeah, but I was thinking of a different approach. Then I remembered that no two dat or sim or whatever files are laid-out the same way, the nodes in the same order or names... so my idea wouldn't work anyway. Might work for the first set of files, but bomb on the next... :salute:

(btw, that is a useful utililty - thank you for that)
Does not matter the lay out of a file. Search for an ID in a broad search? You find it! Trick is to narrow the search to only what your interested in.
Say I want to be sure no 3D model uses the same ID. I search for that ID BUT only in 3D file nodes.

Exactly what do you want to do?

[propbeanie]

I was dreaming of an "all encompassing" utility that could find and fix the files, or even build them - text files in that regard. Use 010 to help me figure out what I'm looking at in the files. I'm kinda dense in that regard though... One thing I'm trying to make use of is the Inspector window in the default interface of 010. Lot's of stuff to explore. :salute:

[Jeff-Groves]

One of the best things to do is to play with the scripting and template ability of 010. Look at the scripts/templates available online at SweetScape.
If one looks like it may be close to your needs? Study how it works.
I'll be starting an aid soon to show you how 010 and S3D will work together to learn the Hex structure of dat files.
S3D shows you a false view. I'm going to show you how to open your eyes.
https://www.youtube.com/watch?v=RhlXqYiTz2Q

[propbeanie]

I know of two guys that would appreciate that! :salute:

[Jeff-Groves]

So you've chosen the red pill.
:haha:

[propbeanie]

In my left hand, yes... Do you have a glass of water though?.. the blue one will be in my right, from when you're not looking and I've already got a purple thing with white polka dots, and another that looks like a gummy bear in my shirt pocket, lint and all, for back-up... :roll:...

[Jeff-Groves]

Once upon a time..............
(Yeah things start with that to this day!)
The only way to work with the files was in Hex.
Most of the early tools did that even if one did not know it was hex work.
Some of the (Actually all the best) Modders worked the files in Hex.
Those were the days before S3D which one person called a "Dumb down tool"
On that I have 2 opinions.
1. Yes.
It did open the door to just being ignorant of the TRUE file structure and hoping it would do EVERYTHING you wanted to do.

2. No.
Blended with a GOOD Hex editor? One can do things in less time than S3D alone can never do.

In Modding SH series of Games always look for the best Tools and ways of doing it. Years of modding with subpar tools have lost a lot of good modders.

[Jeff-Groves]

Been expanding the script to read (False) RAW terrain files in SH5 to now export the information to the Object file format.
That will allow us to SEE the (False) RAW terrain files in a 3D program.
:up:

SH3/4 use a different (True) RAW format then SH5. However. It should be possible to do the same thing for SH3/4 IF the height values
are the same based on info We have.

[Jeff-Groves]

Testing has proven the RAW files are inverted in SH5.
Not surprising in a way. Just have to adjust the scripts now.

[Tycho]

It should be possible to do the same thing for SH3/4 IF the height values are the same based on info We have.
I have sh3 elevations table, extracted from sh3.exe!
Most part look the same like the one posted in "The SH5 EcoMod" thread, but have some differences.
Underwater is different near the shore, and 0-64 band is different. The rest look the same.
When go home tonight, I'll post it if you need it.
I did some experiments there, but now will be interested to change sh3 elevation table with this one from sh5, and to see the result.

[Jeff-Groves]

That information would be useful. We could then create a 3D model of the SH3/4 terrain one could work with in any 3D program.

[Tycho]

SH3 Terrain elevations table

//SH3 Terrain elevations table
//0x530ac0
//0-47 = Underwater
//48-255 = Above-water

// Underwater
0 - 0xc4bb8000 = -1500.0
1 - 0xc3e00000 = -448.0
2 - 0xc3d00000 = -416.0
3 - 0xc3c00000 = -384.0
4 - 0xc3b00000 = -352.0
5 - 0xc3a00000 = -320.0
6 - 0xc3900000 = -288.0
7 - 0xc3800000 = -256.0
8 - 0xc36a0000 = -234.0
9 - 0xc35c0000 = -220.0
10 - 0xc3530000 = -211.0
11 - 0xc34a0000 = -202.0
12 - 0xc3410000 = -193.0
13 - 0xc3380000 = -184.0
14 - 0xc32f0000 = -175.0
15 - 0xc3260000 = -166.0
16 - 0xc31d0000 = -157.0
17 - 0xc3140000 = -148.0
18 - 0xc3090000 = -137.0
19 - 0xc3000000 = -128.0
20 - 0xc2f00000 = -120.0
21 - 0xc2de0000 = -111.0
22 - 0xc2d00000 = -104.0
23 - 0xc2c20000 = -97.0
24 - 0xc2b40000 = -90.0
25 - 0xc2aa0000 = -85.0
26 - 0xc2a00000 = -80.0
27 - 0xc2980000 = -76.0
28 - 0xc2900000 = -72.0
29 - 0xc2880000 = -68.0
30 - 0xc2800000 = -64.0
31 - 0xc2700000 = -60.0
32 - 0xc2600000 = -56.0
33 - 0xc2500000 = -52.0
34 - 0xc2400000 = -48.0
35 - 0xc2300000 = -44.0
36 - 0xc2200000 = -40.0
37 - 0xc2140000 = -37.0
38 - 0xc2080000 = -34.0
39 - 0xc1f80000 = -31.0
40 - 0xc1e00000 = -28.0
41 - 0xc1d00000 = -26.0
42 - 0xc1c00000 = -24.0
43 - 0xc1b00000 = -22.0
44 - 0xc1a00000 = -20.0
45 - 0xc1900000 = -18.0
46 - 0xc1880000 = -17.0
47 - 0xc1800000 = -16.0

// Above-water
48 - 0x40800000 = 4.0
49 - 0x40a00000 = 5.0
50 - 0x40c00000 = 6.0
51 - 0x41000000 = 8.0
52 - 0x41200000 = 10.0
53 - 0x41400000 = 12.0
54 - 0x41600000 = 14.0
55 - 0x41700000 = 15.0
56 - 0x41800000 = 16.0
57 - 0x41900000 = 18.0
58 - 0x41a00000 = 20.0
59 - 0x41b00000 = 22.0
60 - 0x41c00000 = 24.0
61 - 0x41d00000 = 26.0
62 - 0x41e00000 = 28.0
63 - 0x41f00000 = 30.0
64 - 0x42000000 = 32.0
65 - 0x42080000 = 34.0
66 - 0x42100000 = 36.0
67 - 0x42180000 = 38.0
68 - 0x42200000 = 40.0
69 - 0x42280000 = 42.0
70 - 0x42300000 = 44.0
71 - 0x42380000 = 46.0
72 - 0x42400000 = 48.0
73 - 0x42480000 = 50.0
74 - 0x42500000 = 52.0
75 - 0x42580000 = 54.0
76 - 0x42600000 = 56.0
77 - 0x42680000 = 58.0
78 - 0x42700000 = 60.0
79 - 0x42780000 = 62.0
80 - 0x42800000 = 64.0
81 - 0x42880000 = 68.0
82 - 0x42900000 = 72.0
83 - 0x42980000 = 76.0
84 - 0x42a00000 = 80.0
85 - 0x42a80000 = 84.0
86 - 0x42b00000 = 88.0
87 - 0x42b80000 = 92.0
88 - 0x42c00000 = 96.0
89 - 0x42c80000 = 100.0
90 - 0x42d00000 = 104.0
91 - 0x42d80000 = 108.0
92 - 0x42e00000 = 112.0
93 - 0x42e80000 = 116.0
94 - 0x42f00000 = 120.0
95 - 0x42f80000 = 124.0
96 - 0x43000000 = 128.0
97 - 0x43040000 = 132.0
98 - 0x43080000 = 136.0
99 - 0x430c0000 = 140.0
100 - 0x43100000 = 144.0
101 - 0x43140000 = 148.0
102 - 0x43180000 = 152.0
103 - 0x431c0000 = 156.0
104 - 0x43200000 = 160.0
105 - 0x43240000 = 164.0
106 - 0x43280000 = 168.0
107 - 0x432c0000 = 172.0
108 - 0x43300000 = 176.0
109 - 0x43340000 = 180.0
110 - 0x43380000 = 184.0
111 - 0x433c0000 = 188.0
112 - 0x43400000 = 192.0
113 - 0x43480000 = 200.0
114 - 0x43500000 = 208.0
115 - 0x43580000 = 216.0
116 - 0x43600000 = 224.0
117 - 0x43680000 = 232.0
118 - 0x43700000 = 240.0
119 - 0x43780000 = 248.0
120 - 0x43800000 = 256.0
121 - 0x43840000 = 264.0
122 - 0x43880000 = 272.0
123 - 0x438c0000 = 280.0
124 - 0x43900000 = 288.0
125 - 0x43940000 = 296.0
126 - 0x43980000 = 304.0
127 - 0x439c0000 = 312.0
128 - 0x43a00000 = 320.0
129 - 0x43a40000 = 328.0
130 - 0x43a80000 = 336.0
131 - 0x43ac0000 = 344.0
132 - 0x43b00000 = 352.0
133 - 0x43b40000 = 360.0
134 - 0x43b80000 = 368.0
135 - 0x43bc0000 = 376.0
136 - 0x43c00000 = 384.0
137 - 0x43c40000 = 392.0
138 - 0x43c80000 = 400.0
139 - 0x43cc0000 = 408.0
140 - 0x43d00000 = 416.0
141 - 0x43d40000 = 424.0
142 - 0x43d80000 = 432.0
143 - 0x43dc0000 = 440.0
144 - 0x43e00000 = 448.0
145 - 0x43e80000 = 464.0
146 - 0x43f00000 = 480.0
147 - 0x43f80000 = 496.0
148 - 0x44000000 = 512.0
149 - 0x44040000 = 528.0
150 - 0x44080000 = 544.0
151 - 0x440c0000 = 560.0
152 - 0x44100000 = 576.0
153 - 0x44140000 = 592.0
154 - 0x44180000 = 608.0
155 - 0x441c0000 = 624.0
156 - 0x44200000 = 640.0
157 - 0x44240000 = 656.0
158 - 0x44280000 = 672.0
159 - 0x442c0000 = 688.0
160 - 0x44300000 = 704.0
161 - 0x44340000 = 720.0
162 - 0x44380000 = 736.0
163 - 0x443c0000 = 752.0
164 - 0x44400000 = 768.0
165 - 0x44440000 = 784.0
166 - 0x44480000 = 800.0
167 - 0x444c0000 = 816.0
168 - 0x44500000 = 832.0
169 - 0x44540000 = 848.0
170 - 0x44580000 = 864.0
171 - 0x445c0000 = 880.0
172 - 0x44600000 = 896.0
173 - 0x44640000 = 912.0
174 - 0x44680000 = 928.0
175 - 0x446c0000 = 944.0
176 - 0x44700000 = 960.0
177 - 0x44780000 = 992.0
178 - 0x44800000 = 1024.0
179 - 0x44840000 = 1056.0
180 - 0x44880000 = 1088.0
181 - 0x448c0000 = 1120.0
182 - 0x44900000 = 1152.0
183 - 0x44940000 = 1184.0
184 - 0x44980000 = 1216.0
185 - 0x449c0000 = 1248.0
186 - 0x44a00000 = 1280.0
187 - 0x44a40000 = 1312.0
188 - 0x44a80000 = 1344.0
189 - 0x44ac0000 = 1376.0
190 - 0x44b00000 = 1408.0
191 - 0x44b40000 = 1440.0
192 - 0x44b80000 = 1472.0
193 - 0x44bc0000 = 1504.0
194 - 0x44c00000 = 1536.0
195 - 0x44c40000 = 1568.0
196 - 0x44c80000 = 1600.0
197 - 0x44cc0000 = 1632.0
198 - 0x44d00000 = 1664.0
199 - 0x44d40000 = 1696.0
200 - 0x44d80000 = 1728.0
201 - 0x44dc0000 = 1760.0
202 - 0x44e00000 = 1792.0
203 - 0x44e40000 = 1824.0
204 - 0x44e80000 = 1856.0
205 - 0x44ec0000 = 1888.0
206 - 0x44f00000 = 1920.0
207 - 0x44f40000 = 1952.0
208 - 0x44f80000 = 1984.0
209 - 0x45000000 = 2048.0
210 - 0x45040000 = 2112.0
211 - 0x45080000 = 2176.0
212 - 0x450c0000 = 2240.0
213 - 0x45100000 = 2304.0
214 - 0x45140000 = 2368.0
215 - 0x45180000 = 2432.0
216 - 0x451c0000 = 2496.0
217 - 0x45200000 = 2560.0
218 - 0x45240000 = 2624.0
219 - 0x45280000 = 2688.0
220 - 0x452c0000 = 2752.0
221 - 0x45300000 = 2816.0
222 - 0x45340000 = 2880.0
223 - 0x45380000 = 2944.0
224 - 0x453c0000 = 3008.0
225 - 0x45400000 = 3072.0
226 - 0x45440000 = 3136.0
227 - 0x45480000 = 3200.0
228 - 0x454c0000 = 3264.0
229 - 0x45540000 = 3392.0
230 - 0x45600000 = 3584.0
231 - 0x45680000 = 3712.0
232 - 0x45700000 = 3840.0
233 - 0x45780000 = 3968.0
234 - 0x457c0000 = 4032.0
235 - 0x45820000 = 4160.0
236 - 0x45860000 = 4288.0
237 - 0x458a0000 = 4416.0
238 - 0x458e0000 = 4544.0
239 - 0x4594d000 = 4762.0
240 - 0x45960000 = 4800.0
241 - 0x45a20000 = 5184.0
242 - 0x45a92000 = 5412.0
243 - 0x45b12000 = 5668.0
244 - 0x45b6b000 = 5846.0
245 - 0x45be0000 = 6080.0
246 - 0x45c20000 = 6208.0
247 - 0x45c92000 = 6436.0
248 - 0x45d04000 = 6664.0
249 - 0x45d76000 = 6892.0
250 - 0x45db6000 = 7020.0
251 - 0x45e36000 = 7276.0
252 - 0x45e76000 = 7404.0
253 - 0x45ee8000 = 7632.0
254 - 0x45f68000 = 7888.0
255 - 0x45fa0000 = 8000.0

[Jeff-Groves]

Thanks Tycho.
I saved that information for future testing.
Now. The SH3/4 RAW files are 601 x 601 pixels and do use the 0 to 255 colors for height data. Viewing in a 3D program would greatly assist even SH3/4! Testing on the SH5 RAW files is going better then I expected so far.
RAW files from SH5 will read as a 201 x 201 if imported to Photoshop.
THAT IS A FALSE READING!!!
SH5 RAW files are 101 x 101 pixels (Not a very accurate definition).
They are 101 x 101 floats that give the height values of the given area.
MUCH more accurate the SH3/4.

@Tycho
Given you can tell me where to find that data in the SH3.exe? It may be possible to edit for a sharper shore line and maybe shoals effect!

[propbeanie]

This is getting ~really~ good... High hopes! :salute:

[Tycho]

@Tycho
Given you can tell me where to find that data in the SH3.exe? It may be possible to edit for a sharper shore line and maybe shoals effect!
I don't know, probably it is matter of experiment with the elevation values and tests.

This year I decided to take a break from SH3.
Everything related is archived, and will rest for year or two.
So, I will not spend big amount of time and brain power for Silent Hunter.

Here, about terrain I can change this elevations table with sh5 one. Just for experiment, to see the result, and because will be quick and easy.
In fact, in sh3.exe have two such tables, the two are identical.
First one is at: 0x530ac0 /or: 00130ac0
And I saw where is used in the code.
Second one, at: 0x5314f0 /or: 001314f0
I don't saw where is used. Maybe is not used.
It is matter of experiment, are it also need to be changed, if I change the first one.

By address of the first one you can search where is used in the code section.
In this function, for example:


0x506FCE: 53 PUSH EBX ; <==0x00506FC7(*-0x7)
0x506FCF: 33DB XOR EBX,EBX
0x506FD1: 85ED TEST EBP,EBP
0x506FD3: 0F8EA8000000 JLE 0x507081 ; (*+0xAE)
0x506FD9: 56 PUSH ESI
0x506FDA: 57 PUSH EDI
0x506FDB: EB03 JMP 0x506FE0 ; (*+0x5)
0x506FDD: 8D4900 LEA ECX,[ECX]
0x506FE0: 8B44241C MOV EAX,DWORD PTR [ESP+0x1C]; <==0x00506FDB(*-0x5), 0x00507079(*+0x99)
0x506FE4: D90498 FLD DWORD PTR [EAX+EBX*4]
0x506FE7: D9542410 FST DWORD PTR [ESP+0x10]
0x506FEB: D81DC00E5300 FCOMP DWORD PTR [0x530EC0]; .rdata:0x00 0x80 0xBB 0xC4
0x506FF1: DFE0 FNSTSW AX
0x506FF3: F6C441 TEST AH,0x41
0x506FF6: 7A04 JPE 0x506FFC ; (*+0x6)
0x506FF8: 32C0 XOR AL,AL
0x506FFA: EB77 JMP 0x507073 ; (*+0x79)
0x506FFC: D9442410 FLD DWORD PTR [ESP+0x10]; <==0x00506FF6(*-0x6)
0x507000: D81DA8AA5200 FCOMP DWORD PTR [0x52AAA8]; .rdata:0x00 0x00 0xFA 0x45
0x507006: DFE0 FNSTSW AX
0x507008: F6C401 TEST AH,0x1
0x50700B: 7504 JNZ 0x507011 ; (*+0x6)
0x50700D: 0CFF OR AL,0xFF
0x50700F: EB62 JMP 0x507073 ; (*+0x64)
0x507011: D9442410 FLD DWORD PTR [ESP+0x10]; <==0x0050700B(*-0x6)
0x507015: D81DACDA5100 FCOMP DWORD PTR [0x51DAAC]
0x50701B: DFE0 FNSTSW AX
0x50701D: F6C441 TEST AH,0x41
0x507020: 7A0C JPE 0x50702E ; (*+0xE)
0x507022: BFC00A5300 MOV EDI,0x530AC0 ; .rdata:0x00 0x80 0xBB 0xC4
0x507027: BE2F000000 MOV ESI,0x2F
0x50702C: EB0A JMP 0x507038 ; (*+0xC)
0x50702E: BF800B5300 MOV EDI,0x530B80 ; .rdata:0x00 0x00 0x80 0x40 ; <==0x00507020(*-0xE)
0x507033: BECF000000 MOV ESI,0xCF
0x507038: D9442410 FLD DWORD PTR [ESP+0x10]; <==0x0050702C(*-0xC), 0x00507063(*+0x2B)
0x50703C: 8D46FF LEA EAX,[ESI-0x1]
0x50703F: 99 CDQ
0x507040: 2BC2 SUB EAX,EDX
0x507042: 8BC8 MOV ECX,EAX
0x507044: D1F9 SAR ECX,1
0x507046: D81C8F FCOMP DWORD PTR [EDI+ECX*4]
0x507049: DFE0 FNSTSW AX
0x50704E: 7A05 JPE 0x507055 ; (*+0x7)
0x507050: 8D7101 LEA ESI,[ECX+0x1]
0x507053: EB0B JMP 0x507060 ; (*+0xD)
0x507055: 83CAFF OR EDX,0xFF ; <==0x0050704E(*-0x7)
0x507058: 2BD1 SUB EDX,ECX
0x50705A: 8D7C8F04 LEA EDI,[EDI+ECX*4+0x4]
0x50705E: 03F2 ADD ESI,EDX
0x507060: 83FE01 CMP ESI,0x1 ; <==0x00507053(*-0xD)
0x507063: 7FD3 JG 0x507038 ; (*-0x2B)
0x507065: 8B4C2414 MOV ECX,DWORD PTR [ESP+0x14]
0x507069: 8BC7 MOV EAX,EDI
0x50706B: 2DC00A5300 SUB EAX,0x530AC0 ; .rdata:0x00 0x80 0xBB 0xC4
0x507070: C1F802 SAR EAX,0x2
0x507073: 88040B MOV BYTE PTR [EBX+ECX],AL; <==0x00506FFA(*-0x79), 0x0050700F(*-0x64)
0x507076: 43 INC EBX
0x507077: 3BDD CMP EBX,EBP
0x507079: 0F8C61FFFFFF JL 0x506FE0 ; (*-0x99)
0x50707F: 5F POP EDI
0x507080: 5E POP ESI
0x507081: 5B POP EBX ; <==0x00506FD3(*-0xAE)
0x507082: 8BC1 MOV EAX,ECX
0x507084: 5D POP EBP
0x507085: 83C408 ADD ESP,0x8
0x507088: C3 RET


Maybe is possible, let's say, to prioritise the underwater and to simplify the above water terrain. If give it more indexes.
Of course, this will require completely new map and elevations table, if now give 208 to the underwater and 48 to the ground:
The most of the world will go underwater (Sci-Fi mod: Waterworld :hmmm:).
To better understand it, I need to run it through debugger, but I will not.
Maybe is possible, this 601x601 also to be changed, I think, I once saw this somewhere there.

So, probably many things are possible if someone have desire to do it.

[Jeff-Groves]

Tycho,

Thanks Mate!
:salute:

On taking a break? I myself do so from time to time. gap got me interested in the current 010 script to extract the information in the SH5 RAW files.
So far it will extract the heights and create an obj file that one can edit in any 3D program that imports the obj format.

A second script will convert the obj back to a RAW file.

Once the scripts are working as planned? A stand alone program will be done.

[Jeff-Groves]

The start script to export SH5 terrain RAW files to the obj format.
This does all the x y z stuff but does not include the VT or faces code.
Nice looping example thought!
_____________________________________

//------------------------------------------------
//--- 010 Editor Script File
// RAW to object file .003
// Reads the DetailMask RAW files in SH5 and outputs an Object file
//
//------------------------------------------------
int i, l, t, s=101;
char W[40];


float x, y, z;
x=-2550;

for( i = 0; i < s; i++)
{
x=(x+50);
z=-2550;
l=101;

{
for( t = 0; t < l; t++)
{
y = ReadFloat();
z=(z+50);
Printf( "v %f %f %f\n", z, y, x );
FSkip(4);

}
}
}
Printf("Test");// all the code for textures and faces will follow
W = InputString("Save File", "Name your save file", "c:\\temp\\");// You can change the path when saving.
OutputPaneSave( W );

[Jeff-Groves]

Had to send a possible bug report to SweetScape today!
:doh:

Run this script and you'll see the problem at line 84 or so.
This does not interfere with the stand alone C++ version I'm working on.
It only interferes with rapid prototype coding.
Given this error? It may show up as I finalize coding the faces stuff.


//------------------------------------------------
//--- 010 Editor Script File
// RAW to object file add in
// Adds the Textures
//
//------------------------------------------------
// all the code for textures coords below
int j, m, q=101, p;
float vt1=-0.00100000000003808065, vt2;






for( j = 0; j < q; j++)
{

vt1=(vt1+0.001);
vt2=-0.01;
m=101;

{
for( p = 0; p < m; p++)
{
vt2=(vt2+0.01);
Printf( "vt %f %f\n", vt1, vt2);

}
}
}
Printf("texture Testing. Error at texture coord 84");

[Jeff-Groves]

Fixed script above by replacing float with double.

float vt1=-0.00100000000003808065, vt2;
to
double vt1=-0.00100000000003808065, vt2;

and changed the Printf function

Printf( "vt %f %f\n", vt1, vt2);
to
Printf( "vt %lf %lf\n", vt1, vt2);


Will triple check the output then move on to the faces scripting.

[propbeanie]

Ah-HAA! So I'm not the only one with data type issues... shneaky little data bits... Glad you got that. Looking at the documentation, a Double is a "64-Bit Floating Point Number ", while a Float is a "32-Bit Floating Point Number", which with the files of the game, it would seem to make more sense to use a 32-bit data type, but... I suppose it has to do with the OS maybe?? :hmmm: - there are only the 3 types of 64-bit data types, the other two being "int". Does your earlier issue have to do with the rounding factor maybe?? Truncating the data maybe?? :hmmm: Thanks :salute:

[Jeff-Groves]

It's a problem with floats.
From Graeme at SweetScape............

"The error you are seeing is caused because 'float' is not very accurate for doing a lot of computations. We tried your function in C++ and it gives the same issue. 'float' is just not very accurate and if you switch to 'double' it will be better, although double has accuracy issues as well for lots of computations."

Then next section (Faces) uses all ints so shouldn't have a problem.
:)

One thing I am doing is writing separate scripts for each part.
Verts, Vertical Textures, Faces are all separate scripts.
Makes catching errors easier.
The 1st script calls the next 2 in order just as if it was all one script.

[Jeff-Groves]

How to code the faces output.
Given there are 10000 faces? Each line must follow the formula below.
The loop will be slow but should work.
This is NOT actual code but gives the information to code the needed loop.

---C1------ C2------ C3---- C4
f 1/101 102/100 103/201 2/202
f 2/202 103/201 104/302 3/303

C1 = +1/+101
C2 = +1/+101
C3 = +1/+101
C4 = +1/+101

[Jeff-Groves]

At first look you may think we need a double loop like for the Textures.
Actually a single loop will work for our needs so speed goes up.

Look close at the faces ints and the pattern.
You may think 8 ints are needed.

[propbeanie]

It's a problem with floats.
From Graeme at SweetScape............

"The error you are seeing is caused because 'float' is not very accurate for doing a lot of computations. We tried your function in C++ and it gives the same issue. 'float' is just not very accurate and if you switch to 'double' it will be better, although double has accuracy issues as well for lots of computations."
...
Thanks for that Jeff :salute:

[Jeff-Groves]

The support from SweetScape is one big reason I pay to stay updated.
:yeah:

[Jeff-Groves]

Have you figured out the loop needed to output 10000 faces yet?
And the MINIMUM number of ints needed to do it?
:haha:

If your answer is above 3 ints?
:o
Your wrong!
:03:

[propbeanie]

Is there a way to do like a "Do Loop Until EOF"? :salute:

[Jeff-Groves]

We only need one looping function to do everything.
I have 3 ints assigned in my loop right now.
2 only control the number of loops I do.
The other is for kicking out the faces information.
I take advantage of the Printf function to do the math.
:03:

Here's an example..............

f=(f+1);
Printf("f %u", f);
Printf( "/%u ", f=(f+100) );

Those lines print out 1/101
That's the first part of a face.

Each line of Printf needs to be coded or numbers go real FUBAR!
What I mean by that is that using f on a single line in Printf can corrupt the data.
By using different lines? It preserves the true value of f for further math.
BUT! We can loop the code to print out all 10000 lines easily.

[propbeanie]

Nice. I keep confusing "program code" and "scripting"... In other words, I want to write an application in one fell swoop. You're getting the data and its structure, to be used in an app... baby steps. must take baby steps pb... :salute:

[Jeff-Groves]

010 uses a C 'LIKE' code. That means it is not a fully functioning C compliant scripting language.

So it does have it's quirks.

[Jeff-Groves]

OK. I lied on the ints needed.
You only need 1 int.
:har:

I'm going to introduce you to Arrays so you understand why I only need 1 int.

//------------------------------------------------
//--- 010 Editor Script File
// RAW to object file add in
// Adds the Faces
//
//------------------------------------------------
// all the code for Faces below

int f[5];
f[1]=10000;





for (f[2] = 0; f[2] < f[1]; f[2]++)
{

f[3]=(f[3]+1);
Printf("f %u/%u\n ", f[3], f[4]=(f[3]+100));

}





It's a cheat I'll admit. But allows a single Printf function to print the information with no corruption of data.

[propbeanie]

Say uh, does 010 count from zero for arrays (actually, anything), like ANSI C? :salute:

[s7rikeback]

Hello Jeff,


Would it be possible for 010 to list all T0 / A0 / B0 nodes in each dat, within a folder, like Air or Sea ?


So output could be:
Unit Name T01 / T02 / A01 / B01 / B02 etc.


Without having to open each dat with S3D and count them manually ?


Or a template, so that we could enter our own search functions ?

[Jeff-Groves]

The findinfiles function will do that.
Granted you'd need to add some coding to jump past any T01 dds references.

[Kpt. Lehmann]

01000110 01110010 01100101 01100001 01101011 01101001 01101110 00100111 00100000 01010100 01100101 01100011 01101000 01101110 01101111 01101101 01100001 01100111 01100101 01110011 00100001

:D:doh::03:


:arrgh!:

[Jeff-Groves]

Thanks Kpt. Lehmann
Us Freaking Technomages will carry on.
:03:

I saw a post about removing Map Contacts.
In that post it was stated how much work that would be for several hundreds of ships.

So......
Shall I do a script to demonstrate how that can be done?
:03:

[Kpt. Lehmann]

Thanks Kpt. Lehmann
Us Freaking Technomages will carry on.
:03:

I saw a post about removing Map Contacts.
In that post it was stated how much work that would be for several hundreds of ships.

So......
Shall I do a script to demonstrate how that can be done?
:03:

Why yes.... yes you should! :)

Then, you should also do a script to make BACON quadruple itself.... because the only thing better than bacon.... is MORE bacon. :D

[propbeanie]

Did somebody say ~BACON~??!!!??? ee-yum!

[propbeanie]

In support of the "More BACON" principal, Sweetscape is having a "Black Friday" sale on 010 Editor, at 30% off until the 29th. This makes an "upgrade" $15US, or a "new" purchase at $35US for "home" users. Rather nice price point. :salute:

[Jeff-Groves]

One is a FOOL to not take advantage of this Sale!
:D

[propbeanie]

Did you see that they now have a disassembler in it? (I upgraded from v9.) That's for dummy's like me that no longer remember what anything looks like beyond 8 bits... lol " uh... zero 0000, one 0001, two 0010, uh, three 0011, uh.... what's after three?
never mind... four bits... and I'm supposed to be able to count to 15??... I'd get a "F" on that paper... :O:
:roll: :D

[astvitaliy1982]

I don't know, probably it is matter of experiment with the elevation values and tests.

This year I decided to take a break from SH3.
Everything related is archived, and will rest for year or two.
So, I will not spend big amount of time and brain power for Silent Hunter.

Here, about terrain I can change this elevations table with sh5 one. Just for experiment, to see the result, and because will be quick and easy.
In fact, in sh3.exe have two such tables, the two are identical.
First one is at: 0x530ac0 /or: 00130ac0
And I saw where is used in the code.
Second one, at: 0x5314f0 /or: 001314f0
I don't saw where is used. Maybe is not used.
It is matter of experiment, are it also need to be changed, if I change the first one.

By address of the first one you can search where is used in the code section.
In this function, for example:


0x506FCE: 53 PUSH EBX ; <==0x00506FC7(*-0x7)
0x506FCF: 33DB XOR EBX,EBX
0x506FD1: 85ED TEST EBP,EBP
0x506FD3: 0F8EA8000000 JLE 0x507081 ; (*+0xAE)
0x506FD9: 56 PUSH ESI
0x506FDA: 57 PUSH EDI
0x506FDB: EB03 JMP 0x506FE0 ; (*+0x5)
0x506FDD: 8D4900 LEA ECX,[ECX]
0x506FE0: 8B44241C MOV EAX,DWORD PTR [ESP+0x1C]; <==0x00506FDB(*-0x5), 0x00507079(*+0x99)
0x506FE4: D90498 FLD DWORD PTR [EAX+EBX*4]
0x506FE7: D9542410 FST DWORD PTR [ESP+0x10]
0x506FEB: D81DC00E5300 FCOMP DWORD PTR [0x530EC0]; .rdata:0x00 0x80 0xBB 0xC4
0x506FF1: DFE0 FNSTSW AX
0x506FF3: F6C441 TEST AH,0x41
0x506FF6: 7A04 JPE 0x506FFC ; (*+0x6)
0x506FF8: 32C0 XOR AL,AL
0x506FFA: EB77 JMP 0x507073 ; (*+0x79)
0x506FFC: D9442410 FLD DWORD PTR [ESP+0x10]; <==0x00506FF6(*-0x6)
0x507000: D81DA8AA5200 FCOMP DWORD PTR [0x52AAA8]; .rdata:0x00 0x00 0xFA 0x45
0x507006: DFE0 FNSTSW AX
0x507008: F6C401 TEST AH,0x1
0x50700B: 7504 JNZ 0x507011 ; (*+0x6)
0x50700D: 0CFF OR AL,0xFF
0x50700F: EB62 JMP 0x507073 ; (*+0x64)
0x507011: D9442410 FLD DWORD PTR [ESP+0x10]; <==0x0050700B(*-0x6)
0x507015: D81DACDA5100 FCOMP DWORD PTR [0x51DAAC]
0x50701B: DFE0 FNSTSW AX
0x50701D: F6C441 TEST AH,0x41
0x507020: 7A0C JPE 0x50702E ; (*+0xE)
0x507022: BFC00A5300 MOV EDI,0x530AC0 ; .rdata:0x00 0x80 0xBB 0xC4
0x507027: BE2F000000 MOV ESI,0x2F
0x50702C: EB0A JMP 0x507038 ; (*+0xC)
0x50702E: BF800B5300 MOV EDI,0x530B80 ; .rdata:0x00 0x00 0x80 0x40 ; <==0x00507020(*-0xE)
0x507033: BECF000000 MOV ESI,0xCF
0x507038: D9442410 FLD DWORD PTR [ESP+0x10]; <==0x0050702C(*-0xC), 0x00507063(*+0x2B)
0x50703C: 8D46FF LEA EAX,[ESI-0x1]
0x50703F: 99 CDQ
0x507040: 2BC2 SUB EAX,EDX
0x507042: 8BC8 MOV ECX,EAX
0x507044: D1F9 SAR ECX,1
0x507046: D81C8F FCOMP DWORD PTR [EDI+ECX*4]
0x507049: DFE0 FNSTSW AX
0x50704E: 7A05 JPE 0x507055 ; (*+0x7)
0x507050: 8D7101 LEA ESI,[ECX+0x1]
0x507053: EB0B JMP 0x507060 ; (*+0xD)
0x507055: 83CAFF OR EDX,0xFF ; <==0x0050704E(*-0x7)
0x507058: 2BD1 SUB EDX,ECX
0x50705A: 8D7C8F04 LEA EDI,[EDI+ECX*4+0x4]
0x50705E: 03F2 ADD ESI,EDX
0x507060: 83FE01 CMP ESI,0x1 ; <==0x00507053(*-0xD)
0x507063: 7FD3 JG 0x507038 ; (*-0x2B)
0x507065: 8B4C2414 MOV ECX,DWORD PTR [ESP+0x14]
0x507069: 8BC7 MOV EAX,EDI
0x50706B: 2DC00A5300 SUB EAX,0x530AC0 ; .rdata:0x00 0x80 0xBB 0xC4
0x507070: C1F802 SAR EAX,0x2
0x507073: 88040B MOV BYTE PTR [EBX+ECX],AL; <==0x00506FFA(*-0x79), 0x0050700F(*-0x64)
0x507076: 43 INC EBX
0x507077: 3BDD CMP EBX,EBP
0x507079: 0F8C61FFFFFF JL 0x506FE0 ; (*-0x99)
0x50707F: 5F POP EDI
0x507080: 5E POP ESI
0x507081: 5B POP EBX ; <==0x00506FD3(*-0xAE)
0x507082: 8BC1 MOV EAX,ECX
0x507084: 5D POP EBP
0x507085: 83C408 ADD ESP,0x8
0x507088: C3 RET


Maybe is possible, let's say, to prioritise the underwater and to simplify the above water terrain. If give it more indexes.
Of course, this will require completely new map and elevations table, if now give 208 to the underwater and 48 to the ground:
The most of the world will go underwater (Sci-Fi mod: Waterworld :hmmm:).
To better understand it, I need to run it through debugger, but I will not.
Maybe is possible, this 601x601 also to be changed, I think, I once saw this somewhere there.

So, probably many things are possible if someone have desire to do it.


А если значения глубин будут 0-47 и 150-255, а значения высот 48-149, то мир не должен уйти под воду...)
Если бы значения глубин имели бы свой Terrain.pal а значения высот свой отдельный Terrain.pal, это был бы идеальный вариант...

[Jeff-Groves]

Did you see that they now have a disassembler in it? (I upgraded from v9.) That's for dummy's like me that no longer remember what anything looks like beyond 8 bits... lol " uh... zero 0000, one 0001, two 0010, uh, three 0011, uh.... what's after three?
never mind... four bits... and I'm supposed to be able to count to 15??... I'd get a "F" on that paper... :O:
:roll: :D
You know I keep a constant update to 010!
And I have been playing with the DisAssembler!
:03:

I've worked so much in 010? I have to refocus my brain to program in C++!
I can do things so much FASTER in 010!