Cloning Objects in "Plainer English" by stljeffbb1

[stljeffbb1]

Hello everyone!

I first start out by thanking THE_ORZEL, lurbz, Sansal, Commander1980, Shadow9216, iambecomelife, Canaris, and everyone else who has contributed to the original cloning thread....armed with this info, I was able to figure out how to clone objects! It was a somewhat difficult process, and it has taken me the better part of three weekends to finally get a handle on the whole thing....I'm still a beginner at all of this, but I am happy that I was able to clone a ship (Fishing Boat), and I presume that the other files are similarly constructed, but I'm getting ahead of myself! Lets start out from the very beginning...



Yes, this is what I felt like when I first started....all of these new things to learn about....hex strings, hexadecimal numbers, 3D rendering programs, Alpha Channels, etc. etc. etc.....however, I can attest that this stuff is not that difficult to learn, and I will now help to show you with a step by step process on what you need......

First, you will need to get some programs, many of which are free!

For the nitty gritty hex editing, go to www.download.com and type in Hex Workshop in the search area (and also keep "software" as the default choice for which type of programs you want to look for). Download Hex Workshop 4.23.....

Next, for the 3D rendering part of things (because you may want to see you models before you try them in the game), follow this link: http://www.blender.org/cms/Download.13.0.html and get Blender 2.37, which is a completely free rendering program!

Finally, if you want to make texture edits, try either Saint Paint Studio at www.saintpaint.com and/or JASC Corel Paint Shop Pro 9 once again at www.download.com and then typing "Paint Shop Pro" in the search area. These are only trial versions, and you will have to buy them to continue to use them after their trial periods are up....

There is one more tool you will need, and that is the Lurbz DATNavigator tool that was made by Lurbz himself and will give you much of the information you will need when cloning....get this at http://cursos.usal.es/~felipe/dc/datconvert.zip

In addition to the above programs, you will also want a text based editor like Notepad or Microsoft Word.

OK, assuming you have downloaded all of these programs, and can at least open them, lets begin with some basics about hex string ID's, the essential material that will be manipulated in cloning objects....

:|\

It is not too important that we know exactly how these things work, as most of us are hacks (said in the most endearing terms)anyway, we just want the end product to be good, and we figure this stuff out by trial and error....however, for the uninitiated, the hexadecimal number sequence goes like this:

0=0
1=1
2=2
3=3
4=4
5=5
6=6
7=7
8=8
9=9
A=10
B=11
C=12
D=13
E=14
F=15

(for more about hexadecimal stuff, visit http://en.wikipedia.org/wiki/Hexadecimal)

Hexadecimal is used because it is easier for the computer to understand, and I'll just leave it at that

In SH3, hex string IDs are used to help map out where the program will get its info.....items in SH3 are cloned by slightly manipulating these hex string IDs to make different pathways for the game to follow when retrieving information. For example, those turrets on the KGV don't just appear out of nowhere! the game has to know where to fetch them from so they can be rendered in the game....

The hex string ID in SH3 will almost invariably contain 16 digits made up of eight groups of two like this:

6A 34 02 D2 E3 11 79 CC

The beauty of the editing we will do is that:

1) We only change the first two and last two digits
2) It does not matter what we change them to! Although there are some things I avoid and I will describe why later
3) I don't know exactly how this works, but it works!

....OK, lets start figuring out how all of this works!

(continued…)

[stljeffbb1]

First, lets poke around in some of the SH3 files to see what things look like.....lets start with the same Stuka files that THE_ORZEL mentions in his workshop.....

**NOTE** One should REALLY consider making a couple of copies of the original SH3 Folder....I ASSUME NO RESPONSIBILITY FOR YOU MESSING UP YOUR ORIGINAL COPY OF SH3!!!! Work on the copied versions instead....you can run the copy of that game by clicking on the application icon in your main directory for that folder that looks like:



OK, go to this folder:

C:\Program_Files\Ubisoft\SilentHunterIII\data\Air

In this folder, you will see a bunch of different planes....click on folder "ADB_JU87Stuka" and it will look something like this:



Notice the different types of files like "DAT file," "SIM file," "CFG file," etc......

TO CLONE AN AIRPLANE, YOU WILL NEED TO USE THE FOLLOWING FILE EXTENSIONS:

DAT
ZON
SIM
DSD

IF YOU WERE EDITING A SHIP, YOU WOULD ALSO INCLUDE:

VAL

This is all you basically need to know concerning which files you need when cloning....you will also need to edit the CFG file, and you may wish to edit the EQP file...we will get into this later.....

Now that you know which files you will need to edit, lets start the process.....

First, copy the ENTIRE ADB_JU87Stuka folder and rename it ADB_ClonedJU87Stuka, and rename ALL of the files in the folder ADB_ClonedJU87Stuka.XXX (EDIT: where XXX equals the different file extensions like CFG, DAT, etc....do this for all of them, including the .tga images and the text files).

Then, open up the Lurbz DATNavigator (also known as datconvert).....in the upper right of the window, select "Pick File" and open only the C:\Program_Files\Ubisoft\SilentHunterIII\data\Air\ ADB_ClonedJU87Stuka\ADB_ClonedJU87Stuka.DAT file.....

Now, a note on loading times.....I have seen this program take up to 40 minutes to open the BB Revenge.DAT file....you will need to give this program some patience! Also, I run on a Pentium 4 2.6 Ghz with 1 Gb RAM....if your computer is slower, yours will probably take longer! GIVE IT TIME!!!!!

Once the data has been "navigated," it should look like this:



On the "Hierarchy" portion in the upper left of the window, you will notice a little "+" with 5.-JU-87Stuka_fuselage...go ahead and uncollapse ALL of the branches so it looks like:



OK, after this is done, notice the right hand portion of the window....you will see four slim rectangular boxes that have the following labels:

id
parent
txtrref
3D

At this time, I suggest you open your text editing program (Microsoft Word) and set it up like this:



Each of the slim rectangular bozes contain the ID strings that I was talking about earlier....all you will be doing is using your mouse to select the strings, right click copy, and then paste it on to your text program and start to establish a list like you see I've done...

Now, DO NOT WORRY about the "parent" box on the right hand side of the DATNav window....also, the string under the "txtrref" box will often be the same, so the strings you will ALWAYS copy are the "id" and "3D" strings....to summarize:

id = ALWAYS COPY
parent = NEVER COPY
txtrref = OCCASIONALLY COPY (I will explain)
3D = ALWAYS COPY EDIT 24 Sept 05: Snipped an extra "[/b]"

Concerning the "txtrref" box, it seems that this changes based what version of the object is in effect....this is why there are never more than seven or so different strings for ALL of the different chunks (because in the game, there are the 1939, 1940, through 1945 versions of ships, planes, etc.).....if you want, you can copy every one onto Word, and you can just search and remove the duplicates later. Once all of the strings are copied, you will have to delete ALL of the dashes out of the strings so you can copy them directly onto the Hex Workshop program:

Before:12-34-56-78-90-AB-CD-EF
After: 1234567890ABCDEF

(continued….)

[stljeffbb1]

OK, now, hopefully you have been able to extract all of the IDs that the DATNav tool has provided, however, DATNav is a "messy, dirty" tool as I think he described it....it is great, but it does not always provide all of the string ID's we need.....so now, its time to open up Hex Workshop:

Upon opening this program, open up the following files IN THIS ORDER:

ADB_ClonedJU87Stuka.DAT
ADB_ClonedJU87Stuka.ZON
ADB_ClonedJU87Stuka.SIM
ADB_ClonedJU87Stuka.DSD

(if there were one as there are for ships, you would next open ADB_ClonedJU87Stuka.VAL, but there is not one)

Hex Workshop should look like this:



On the far left of the middle window, you see listings by decimal numbers, but I think this is changeable within the program. The middle portion of the middle window has all of the different hexadecimal strings, and then the right portion of the middle window has the translation into text (which is why when you open up a .dat file with a text reader it looks like gibberish)....

Now for a word on hex string data.....when the programmers made this program, they developed what is called a Software Development Kit which helps them to make the program....however, especially with games that are still viable, computer game companies generally DO NOT like releasing their SDK, however, we can at least see the data arranged in hexadecimal format, because all computer programs use the basic premise of 0=off and 1=on....basically, anything that a computer does is made up of trillions, quadrillions, and even quintillions of on and off switches (if not more!)....because of this, we can translate any program into hexidecimal, but we cannot necessarily make heads or tails of it.....sometimes, programmers throw us a bone by including some text information (seen in the right hand side of the middle window), however, often "hackers" must figure things out by trial and error.....

Back to the Hex Workshop....I suggest you arrange your text file with the copied strings below your Hex Workshop window like this:



OK, now go to the Edit menu on the top toolbar and find the replace command, which can also be obtained by hitting Ctrl-H.



Now, we will begin the cloning process! Start with the first string, right-click copy right into the "Find" line AND the "Replace" line of the Replace window. Now you get to be creative!

To change the string and effect a clone, change the first two and last two digits to anything you desire (within the hexadecimal system)

Example:

First string (from ID 5.-JU-87Stuka_fuselage)

F836293DBF589F8A

can become:

E736293DBF589F22

Notice that the inner digits remain the same...it does not matter what you change the outer 4 digits too! However, I strongly suggest that you avoid using "00" at either the beginning or end of the string, and avoid using a "0" to start or end a string....I will explain why soon....

Now, this is important! When you start with the very first string, start with the DAT tool and replace the first string with your string….after this, select the second tab below the middle window….you won’t able to read it, but if you opened the files as described above, this tab will be the .ZON file…..now, use Ctrl-H (quicker) or go to the Edit menu and select “Replace�….the program is nice and keeps the same thread you just replaced! After getting done, click on the third tab (.SIM) and repeat the Replace procedure, then finally click on the fourth tab (.DSD) and do the same….you will not always have all of the IDs in all of the different (ZON, SIM, DSD) files, so don’t worry if you get the “no string� message.

(continued....)

[stljeffbb1]

OK, so you've now copied all of you strings that DATNav provided into Hex Workshop....don't do ANYTHING quite yet.....here is what it should look similar to:



This was taken from the Liberator.DAT file, and even though THE_ORZEL had provided all of the string IDs, I wanted to see if the DATNav tool would catch them all, and it didn't...

However, if you closely look at the screen, you will see a pattern of red bands (which represents changed data) that is nice and symmetrical, and also some gaps.....

In an earlier thread on THE_ORZEL's thread, I posted that I found out that ALL of the necessary ID strings are located at the bottom of the data on the .DAT file represented in Hex Workshop....this was proven true because I was able to clone the Fishing Boat without having any strings provided on the message boards and having to fill in the missing strings that the DATNav tool does not provide....

"The Art of Finding Strings"

OK, lets take a close look at the Liberator file:



Find the spot where, in the right hand column, the text Liberator_Mrk03 appears....this seems to be a marker. Then, locate the data: "!?J.PH.=.*.?...@.59..." This also appears to be a marker...

Now, remember when I said "don't use '00' to start a thread?" This is because the game typically uses "00" to set apart one string from another. As it turns out, the dark highlighted section in the upper left of the window is the first string in THE_ORZEL's Liberator ID message....I had guessed that if I filled in the ID's that the DaTNav tool had missed by using them provided by THE_ORZEL, the Hex Workshop red bands would look nice and neat....and they do!



BEAUTIFUL!

So, what you have to do is look for similar instances of data....first find the last thing you can read in English in the right hand column....then, look for that little chunk of data not long after this, then, look for data in strings that have "00" right before it and/or look for the pattern that is established by your already entered ID strings from the DATNav tool....not all files have the same kind of markers...the Liberator had either "0C 00" or "0D 00" while the Fishing Boat only had "00" ......


You might wonder....OK, I've added in the strings (from the DATNav), but there are extra data that hangs over the edited parts (not in red)...well, remember I'm new at all of this too, and I don't know about that data, but I guess just don't worry about it?!? I have had good luck so far with planes and a ship....we will see when I attempt to clone a Battleship, however, I'll bet that the same ideas here apply to all ships!

Now that you have found the missing strings and you have them all on your text document (I recommend you type in a line that separates the DATNav strings from the ones you find), save that document (don’t lose it!) and then use the Replace feature in Hex Workshop and do like you did earlier with all of the other files (ZON, SIM, and DSD) to get ALL of the IDs changed that are necessary.

Just for fun, I’ll show you the Stuka.DAT file and where the first ID (I think!) starts:



(continued....)

[stljeffbb1]

Now, you should have all of the ID strings that you need to be replaced in the Hex Workshop on the DAT, ZON, SIM, and DSD files....you are almost done!

In the text Cloned Stuka CFG file, open it up and change the "class" from "DBJU87Stuka" to "DBClonedJU87Stuka"

Now is where things can get interesting....open up the Cloned Stuka EQP file, and you can add different load outs! This is one of the main reasons for cloning objects in SH3....you can keep the integrity of the original plane and add new planes that can be modified to your liking, or even changed to a completely different plane (as sergbuto does!) ...unfortunately, SH3 does not seem to have the airplanes able to fire machine guns from the front of the plane, only the rear turrets will fire

Still, these modifications are for another thread!

Now, go up one level to the Air folder and find the "EnglishNames" and / or "GermanNames" CFG files.....add the line DBClonedJU87Stuka=JU 87 Clone, or whatever you want to call your clone!

After this, go to the Roster folder (on the same level) and you can add this plane to any country you choose, and it is a different Stuka from the stock version!.....simply make a CFG file similar to the others with the proper names....

One last thing, go back to the ADB_ClonedJU87Stuka file and change the "ADB_ClonedJU87Stuka_en" and / or the "ADB_ClonedJU87Stuka_ge" text files to properly represent the plane in the Museum (especially if you have changed the loadouts)

For fun, add the Cloned Stuka to the American Air roster.....

FINALLY.....you should be ready! Fire up the game (using the proper application file based on the files you altered) and go to the museum.....with anticipation, click on the American air roster and see your cloned plane! It should look like:



The game automatically adds the roundels based on the country it fights for!

WOOHOO!!!!!!!!!!!!

Now, if the game CTD, this generally means that you have not properly copied all of the IDs....back to the drawing board

WHEW, now I'm tired! Lots of typing.....I hope this helps lead people in the right direction concerning cloning....as of yet, I have not learned how to actually edit objects after they are cloned, and I know there is a way to add separate textures for each clone...probably have to use the .tga extractor (Chessmaster program is very good....can be found somewhere around here) and find out where it is in the .DAT file then rename it....

I will do my best to answer questions as they come....I will gladly defer to those who know more than I do about all of this...

The main goals of cloning are to:

1) Help fix the holes that SH3 left in missing planes, ships, etc.
2) Help people develop their own objects
3) Help people learn more about how games work

Good Luck!

-Jeff

[sergbuto]

Unfortunately, using such a total cloning method will put higher demands on computer resources and may eventually oveload some systems.

The SH3 engines smartly allows for multiple use of 3D meshes. It is only the object "representation/description" needs to have an unique ID, not the 3D mesh itself. That allows for lower memory consumption without loading multiple copies of identical 3D meshes. Therefore, it is enough to change only IDs given by DAT navigator plus TGA IDs to get a fully functional clone-model in the game. The 3D mesh needs to have the unique ID only when it is to be edited/modified.

[stljeffbb1]

Hi sergbuto.....thanks for the tip!

So, what probably should be done is:

1) make clones that are designed to be used as pure copies of a given unit, meaning that less ID's need to be changed.....

2) then, make other clones with the "full method" that are designed to be edited / modified.....

Also, in discussions with Rubini concerning the B24 Liberator clone, we have determined that my clone comes from a later version of SH3 (with RUb 1.42 and a modified airpower mod), and I will need to clone from "plain" versions of SH3 1.4b to ensure a "pure" clone.

I still need to learn how to find .tga files embedded in the DAT files.....

-Jeff

[Commander1980]

Thank you very much jeff! This is great! When i find more time, i will try some cloning

The dat navigator needs much time, indeed. I have a 2800+ with 1024 MB, and i waited 40 minutes for the C-Class Dat-file...

[iambecomelife]

Hey stljeffbb1 - I noticed you said it took you about 40 minutes to open a file (Revenge). I'm not sure if this is your bottleneck, but try opening them when you don't have any browser windows open. Offline I've opened battleship dat's in about 4-5 minutes even though my specs are similar to yours. If that doesn't work, try closing as many programs as you can.

[stljeffbb1]

Hi iambecomelife....thanks for the tip! I'll try this next time I use DATNav.....

-Jeff

[Shadow9216]

For the cloned liberator, what is the easiest way to change the name as it displays in the museum? I'd like to represent the Navy version of the plane, which became the privateer....which I think we could make if we could add the tail unit from the Sunderland and the waist blisters from the Catalina.

[stljeffbb1]

Hello Shadow.....You may change it in the the folder:

SilentHunterIII\data\Air\EnglishNames.CFG or GermanNames.CFG

There, you will see the class name on the left with the display name on the right.



-Jeff

[iambecomelife]

Speaking of battleships I'm about halfway done with my HMS Renown/Repulse mod. The actual cloning is done, and now I'm doing the skin and armament. With any luck it will be done by this weekend.

[Sansal]

Hi boys:

As a continuation of awesome work of Lurbz ( http://www.subsim.com/phpBB/viewtopi...c&start=50#top ), i'm making a new utility for extract and modify the 3D objects. It isn't finished but one of her functions is clone any unit in the game:

http://rapidshare.de/files/5416649/p...20105.zip.html

It need the JRE of Java http://java.com/

Options:

O: Output all 3D parts and TGA in a subdirecctory named 3D


I: Reinsert a TGA in the dat without the actual restriction of size. Now i'm working in a problem with reinyection of .obj It's important that the three or two prefix name tga was in the format 3_new_name.tga and this number are equal as original tga.


L: List, in few seconds, all chunks in the .dat, .sim, .zon, .val with details of name, size and position in the file.


T: List only id of chunks

C: Clone object. The number can be any long


It no have a GUI (windows interface) but i will make it when all functions are finished. For this reason only work in the Console (MS-DOS) of Windows, and the best form to work with object is copy the programa and files in the same directory, sorry about this.

Cheers,

[Rubini]

Awesome Sansal!

Now we have all the tools!

So, is't ready to use in MS-DOS windows console?

Rubini.