Harbor Editor

[Jeff-Groves]

Just looking into creating a Harbor Editor and wondering if it's something needed?
ref had a crude version once but that was like 15+ years ago?

What I have done so far is.....
Export the Harbor_kit.dat
Write a Script for 010 to grap the ID of everything and make a list of them.
Working on a Script to Rename all files exported to the IDs.

To Do.....
Write a Script to change the Names on Exported files to proper ID.
Write many more Scripts to read the Locations files and build a complete Harbor from them.


Now I have Blender 2.79 set up to do batch imports of Obj files. So that is taken care of.
It's the transitional information I'm working on upon import.

Needless to say You'd need 010 to use My work flow.

Notes:
I used Pack3D to export the Harbor_Kit...........
It's flawed when it comes to textures used by 3D objects. Not much of an issue as We want placement information.

[Jeff-Groves]

Started the Code to read the Terrain Locations dat files so I can Translate the Placements.

Just a quick peek at some of the code.

void setUpRotationMatrix(float angle, float u, float v, float w) {
float L = (u*u + v * v + w * w);
angle = angle * M_PI / 180.0; // converting to radian value
float u2 = u * u;
float v2 = v * v;
float w2 = w * w;
rotationMatrix[0][0] = (u2 + (v2 + w2) * cos(angle)) / L;
rotationMatrix[0][1] = (u * v * (1 - cos(angle)) - w * sqrt(L) * sin(angle)) / L;
rotationMatrix[0][2] = (u * w * (1 - cos(angle)) + v * sqrt(L) * sin(angle)) / L;
rotationMatrix[0][3] = 0.0;
rotationMatrix[1][0] = (u * v * (1 - cos(angle)) + w * sqrt(L) * sin(angle)) / L;
rotationMatrix[1][1] = (v2 + (u2 + w2) * cos(angle)) / L;
rotationMatrix[1][2] = (v * w * (1 - cos(angle)) - u * sqrt(L) * sin(angle)) / L;
rotationMatrix[1][3] = 0.0;
rotationMatrix[2][0] = (u * w * (1 - cos(angle)) - v * sqrt(L) * sin(angle)) / L;
rotationMatrix[2][1] = (v * w * (1 - cos(angle)) + u * sqrt(L) * sin(angle)) / L;
rotationMatrix[2][2] = (w2 + (u2 + v2) * cos(angle)) / L;
rotationMatrix[2][3] = 0.0;
rotationMatrix[3][0] = 0.0;
rotationMatrix[3][1] = 0.0;
rotationMatrix[3][2] = 0.0;
rotationMatrix[3][3] = 1.0;
}

[propbeanie]

You can "build" a pre-Placement dat with S3D Jeff, using Blender to make your "placements" in its grid. The problem comes in the "translation" from the Blender to the S3D coordinate system, in that Blender will "import" obj files "vertically" unless you do an import transposition, and the same for export. The coordinates themselves for X, Y & Z are the same, except the "Y" coordinate will be the opposite between the two, such that a "2.671" in Blender will have to be "-2.671" in S3D, and vice versa. With the Rotation coordinates, you are better off with the "degress" setting in S3D, since that's what is 'native' to Blender. It is also best to not exceed 0 to 180 and 0 to -180 for the rotation settings. After a "harbor" (or whatever) is built in Blender, it almost directly translates into a file for S3D, with the tedium being the hex identifiers for the assets used, as well as the Coordinates and Rotation for each of those. Animations and Sounds can also be placed, but those use a slightly different set-up from the "Placement" nodes in S3D, as you're already familiar with and seen. I was attempting to look at what Swasjer posted on GitHub for his libraries, which might translate well into your 010 scripts, and might actually be able to be used as the basis for a self-contained program... as you mention though, "... wondering if it's something needed?" It would be very nice to have when needed / wanted, but, is it worth the effort to do all this? it is rather recursive to implement, perfect for programming, but it's those "details" that kill ya with this... :arrgh!:

[Jeff-Groves]

Roger on all that propbeanie. :salute:

Now why would I look into doing all that hard work?
I'm retired now and enjoy wasting my time doing things no one else does. Or are less likely to do anyways.
:hmmm:

It's mostly to see IF I can do it for Me.
:D

One thing I hate about S3D is the way it shows the ID in Hex. It's a backwards display compared to what one sees in a hex editor.

Oh. Just so anyone knows? The reason for renaming the obj files to it's ID is because the names found in location dats don't mean much for my needs.
It's the object ID that never changes.
And when you export from Blender? You can change the settings to Y up and more.
:03:

[iambecomelife]

Wow! This stuff goes way over my head - but it is fascinating all the same! Thanks for your continued work for those of us who aren't too familiar with making these changes.

A harbor editor would definitely be nice because even back in the late 2000's SH3 and SH4's harbors were not my favorite .... too similar looking. Now in 2024 they are showing their age even more. In real life of course each harbor has unique breakwaters, orientation of harbor facilities North or South, etc. It would be great to have each harbor look more like it really did historically.


Eagerly looking forward to this! :up:

[Jeff-Groves]

After a lot of thinking on it? I've got most of it worked out in my head.
I'm not good at programing things for 3D viewing but I can manipulate the heck out of the obj format! As 010 does use a type of C coding? I may even make a stand alone program at some point to handle the transformations.
:hmmm:

Now I know just enough about Blender (Even less about Wings3D) to be dangerous! So those that use those programs? I'll need some help.

I can tell you that Pack3D does a good job doing the mass export of needed files. BUT. Since many objects have multipule UVs? It stacks them all into the obj file. But that is not a problem as I only need the verts to do all the Math on.
Moving the objects is just a matter of adding the XYZ from a location.dat file to a given object file. Rotation is more complicated Math but I've already coded that!
The stuff I'm coding will do the heavy lifting work. Once you edit things in a 3D program and export the objects? The codeing will reverse that and spit out a PROPER Location.dat! You'll only need to MAYBE change IDs.
I've got a way to do that automagically also in my head and have posted before on how one can do that!

I've started the code to grab ID's and names of all 3D models in the Harbor_Kit.dat
That will rename all the 3D objects I've exported to the ID's.

Then I'll rip the needed stuff from a Location.dat to get things ready to import to a 3D program. It's pretty intense but I'm haveing a BLAST doing a Jethro Bodine on this!
:har:

Oh Yeah. I'll also be extracting the needed areas with ref's SH3 Tool and converting that information to an object file! Making it easier to do a Harbor or other location. Already did that kind of tool for SH5.

[propbeanie]

Combination of a "tease" and what you can do with the HarborKit dat file "assets" exported with S3D and brought into Blender:

https://i.imgur.com/s8ywBPN.jpg


Lots of "parts" missing, lots of "overlaps", etc., but the Placement nodes in S3D can be built or populated from the Blender Location and Rotation slots. We are currently working on trying to get a 2D rendering of the docks for a reasonably accurate placement in the SH4 game, but the same can be done with the SH3 Locations.cfg file in the Terrain folder. It's just more frustrating to work in SH3 in this regard...

:salute:

[propbeanie]

... Moving the objects is just a matter of adding the XYZ from a location.dat file to a given object file. Rotation is more complicated Math but I've already coded that!
The stuff I'm coding will do the heavy lifting work. Once you edit things in a 3D program and export the objects? The codeing will reverse that and spit out a PROPER Location.dat! You'll only need to MAYBE change IDs.
I've got a way to do that automagically also in my head and have posted before on how one can do that!

I've started the code to grab ID's and names of all 3D models in the Harbor_Kit.dat
That will rename all the 3D objects I've exported to the ID's.
...
Cool beans Maynard! especially that ID and names of the 3D models... for the above, I had to search the name, verify I had the intended "asset", and then copy the node's ID to use for the Placement. Nothing difficult per se, but definitely something a script can do much more efficiently that this old brain fahrt can... :arrgh!:

[Jeff-Groves]

Having Blender open and S3D to get all the information is a Super PITA!
And hours/Days/Weeks to do!

I plan to have a Brand New Location.dat done is a day or 2.
:03:

How are you getting the XYZ and Rotation information from Blender?

[propbeanie]

With your Blender project open, you select one of the 3D items, such as that "150m12.5_dig" showing in the "Collection" in the right-hand pane. You then select (if it doesn't default to it) the "Object Properties", which is that little orange solid box inside the brackets there between the 3D window, and the lower right-hand pane. That will have the X, Y and Z Location, as well as the X, Y and Z Rotation. Those numbers can be used directly in the Placement node. The problem comes in with the differences between the S3D (maybe Pack3D also) obj "orientation". S3D like to export the 3D obj "vertically", like an airplane or ship on its tail, whereas Blender likes things horizontally, on its wheels. It's been too long since I worked with something exported with Pack3D, so I do not remember its orientation. Anyway, suffice to say, you might have to reverse the Y and Z Locations and/or Rotations, and possibly have to convert from a negative number to a positive for one of the Location settings. Besides all that "looking things up manually", experimenting with the Placement build in the game is almost always necessary. So view it in-game before going much beyond a few of the obj Object Properties, or you might find you end up with a jumbled mess, like I had for a good while. Sometimes also, a person has a tendency to over-think things though, and combined with covid brain, it can get ugly... :doh:

[Jeff-Groves]

Well my Good Friend propbeanie,

I find that information Great! But you know Me. I can write code around things like that. Any help you and others give is welcome!

[Mister_M]

Hum. Maybe it would be far easier to build a harbour directly into the Harbour_kit.dat, and create the harbour file (in locations) with only ONE placement node which points to this harbour... So, the Harbour_kit.dat works like a harbour editor. :)

[Jeff-Groves]

Mister_M, Yes it can be done that way. I'm not interested in doing it that way though. And that does not help to Edit existing Harbors.

Thanks to propbeanie I've located the built in move and rotate panel.
I'm not a Blender user so I have to learn things. One thing I've learned is Running a script in Blender. Now that is exactly what I plan to do.
I'll write a script that will do the relocation of objects in Blender in Mass.
The Scripts for 010 will read the needed information then spit out a complete script for Blender.

:)

[Jeff-Groves]

Got the Script that will move an object and rotate it in degrees figured out.
And a batch import handled.

Now I have to keep studying to figure out how to apply any transforms to objects by name after they are imported.
Still learning the Blender Python APIs

What I want to do is after a batch import, run the script to adjust the objects by name.
So if theres 200 objects they all would be moved and rotated based on what the script provides.

Oh yeah. I can only run Blender 2.79 since I still work with WhinDoz 7
Until I get a New system I could probably load a newer version on SWMBO's WhinDoz 11 contraption.
It's got a Killer Ryzen 7 CPU!

So here's a starter script I run in Blender 2.79 to mass import. I'll be working on that to do any transforms as objects are imported.


import bpy
import glob
import os

importDir = "E:\\Desktop\\3D\ "
print(importDir)


os.chdir(importDir)
for files in glob.glob("*.obj"):
print( files )
bpy.ops.import_scene.obj(filepath=files, filter_glob="*.obj;*.mtl")

That Script imports every object that I exported from the Harbor_Kit.dat!
Now I just have to intercept each object and apply the Transition code stuff as it imports.

Guess I'll have to kind of load a Newer Blender on SWMBO's 'Puter and test it?
Unless a kind person does it for me? You can change the importDir to whatever works for you.

[Mister_M]

I hope you know object-oriented programming... it will be useful! :)

[Jeff-Groves]

I can program in C, C++, Python, and a few others. So I do know object-oriented programming.
:03:

Downloaded Blender 4.3.1 Portable to play on SWMBOs whinedoh 11 Ryzen puter.
I run it off a Portable Thumb Drive. Gotta say it is different.

[Jeff-Groves]

Getting real close to doing what I want for Blender.
Time to work on the stuff to grab the info I want from a location.dat and mix that into a Blender script.

Gonna work on doing the Terrain file stuff first as I think it's needed to fill things out.

I also take time to help others with stuff. Looking at Grenso's issue and spent time with Mister_M teaching him a few things.
:up:

[s7rikeback]

Getting real close to doing what I want for Blender.
Time to work on the stuff to grab the info I want from a location.dat and mix that into a Blender script.

Gonna work on doing the Terrain file stuff first as I think it's needed to fill things out.

I also take time to help others with stuff. Looking at Grenso's issue and spent time with Mister_M teaching him a few things.
:up:

Looking forward to seeing how this goes Jeff. This could open up a lot of possibilities with harbor designs and features.

[Jeff-Groves]

Thanks Mate. Just wish I'd have looked at Blender long ago!
The scripting ability is amazing! I put it right up there with 010!
:up:

May even remove 3DS Max from my system.

[propbeanie]

My youngest embarrasses me with his Blender skillz... he is doing 3D modeling AND animation. I am always asking him questions. Understanding the answer is my main problem... :doh: - plus, there is a certain "old head" modder we all know and love, who just started in Blender and 3D modeling less than a year ago and surpassed me a while back... of course, I am like the turtle in the hare race, except I have to stop and watch ToobYu vidz for just about everything I try to do... my skull is like a lobster trap that lets the brains leak out... :D

[Jeff-Groves]

My youngest embarrasses me with his Blender skillz... he is doing 3D modeling AND animation. I am always asking him questions. Understanding the answer is my main problem... :doh: - plus, there is a certain "old head" modder we all know and love, who just started in Blender and 3D modeling less than a year ago and surpassed me a while back... of course, I am like the turtle in the hare race, except I have to stop and watch ToobYu vidz for just about everything I try to do... my skull is like a lobster trap that lets the brains leak out... :D

I'm just getting started with writing scripts for Blender! Still learning to actually work with it.
For what I have learned so far? I can see doing a script to just open a Location.dat and have it load with no other work needed!

[propbeanie]

If a program is written, I can see the user wanting a visual "catalog" of all that is available... "Let's see, I want the green one - no, make that red... changed my mind - I like the blue the best... maybe..." :arrgh!:

[Jeff-Groves]

I suppose you'd like it to work with touch Screen also?
:haha:

I do have an old project that uses Direct3D11 for touch screen code.
:hmmm:

Back to doing a Script........
I'm thinking a multidimensional array would work to adjust obj files on import.
So say the Location.dat has 200 objects. I'd have the array set for 200 by 3 or 6 depending on Blender Python.

[Jeff-Groves]

Spending a LOT of hours just reading up on Blender Python.
To find what I need I use 010 to search the files in Blender.
I was confused about Numpy. Numpy is a math library for Python.
I need that to create a multidimensional array as basic Python does not provide for those. Fortunately Blender has Numpy which took me a while figuring out since searches on the weebz talked about installing it. :roll:

Now I read the Numpy Python stuff in Blender for proper usage.
:O:

And from reading those files? It appears that it can be used to read a Location.dat file directly! Thus removing the need for 010 to grab the basic information.
:hmmm:

[Jeff-Groves]

Was gonna post a new image but seems image posting is all FUBAR?
:hmmm:
Ya wanna see it go here....

https://imgur.com/02EEGIH

[propbeanie]

Oooh-ooh... I see an outline of a dock there... nice

https://i.imgur.com/02EEGIH.jpeg


as for the images, getting the correct link can be a bit tough with imgur... or any of the sites anymore, especially if you're doing a gif. They want to sling ads at everybody. :salute:

[Jeff-Groves]

That image is placement nodes for the Djakarta.dat location file.
:03:

[Mister_M]

Any news on that work? I think it could be useful to improve the forests too. :03: