SUBSIM Radio Room Forums - [WIP] Almagest. Strict Import for SH5.

SUBSIM Radio Room Forums (https://www.subsim.com/radioroom/index.php)

- SH5 Mods Workshop (https://www.subsim.com/radioroom/forumdisplay.php?f=249)

- - [WIP] Almagest. Strict Import for SH5. (https://www.subsim.com/radioroom/showthread.php?t=248714)

Well. Did some code modifications and just could not get the proper out put!
Spent several days and wasted those days!

Seems I Forgot that the basic code looks for a certain file in a very specific location!
:/\\!!

I haven't added the UI to select a file yet.
:oops:

Once I placed the file and named it properly? It worked perfectly.

I've also to add a routine so one can select if this is for TDW's SH5 import or Skywas's S3D import.
There are different things to consider for each.

I did move the Code to Visual Studio 2010 Express C++.
Not sure if that is a smart move as yet but We will see.

So, things are nearly finished ? :D

Quote:

Originally Posted by Jeff-Groves (Post 2786597)

There you go!
You looked behind the curtain and saw the Magic.

Hahahah, as of now I did not see any magic, sorry, but I am trying to figure out the logic and I must add that, complicated as it is with all those v, vt, vn and f lines to track down, most of what I see in the obj format finally makes sense to me.

Now I know that everything must be in the right order for us to be sure that UV2 coordinates are imported correctly in game. I know what I did wrong in the past and how to avoid those mistakes. Even so, the error is around the corner. A single, little mistake can mess up a whole UV map. For this reason, a program tracking down the most common causes of secondary UV map corruption, and correcting them where possible, would still be a valid help :yep:

From this perspective, Blender is an useful editing and diagnostic tool, as it allows us to import the UV map from one model to another identical model. If the two models have the same geometry but v/vt sorted in a different order, it will be immediately apparent that the imported UV map is buggy, and so it will be in game. I think kapuhy exploited this feature, and this is why he never had similar problems importing his AO-mapped units in game.
Nonetheless, Blender can't debug wrong AO models created with other programs, and this is where Almagest might show its AutoMagic powers :|\\ :D

Quote:

Originally Posted by Jeff-Groves (Post 2786597)

I do suggest using Blender files as they are way smaller but produce the same results! Plus you get the modifier you like.

That's something I have noticed too. I believe Belnder to store the same information as Wings3D, but in a more "compact" way. That shouldn't have any effect on GR2 file size though.

There is a con to Blender-exported obj files though. If memory serves, GR2 Editor doesn't recognize one of the ASCII character it uses as line separator. If you open one of those files in Mircrosoft notepad you will immediately notice the difference as all the lines are shown one after the other, as if they were in the same paragraph. For GR2 Editor to read those files one needs to export/re-export the files in another program, or Notepad++ must be used to replace one character with the other. Kapuhy might know more on this topic.

Quote:

Originally Posted by Jeff-Groves (Post 2786597)

Not sure WHY the Wings files have so many Normals compared to the Blender files?
:hmmm:

If you are referring to the files I sent you the answer is easy: I think my Blender exports have as many normals as vertices because I split hard edges before exporting. Provided that this is the correct answer, Wings-exported files have not a bigger number of vertex normals compared to Blender exports, but the other way around: they have a lesser number of vertices.

On the differences of Wings and Blender files.

The hexadecimal 0a, a control character as opposed to a printing character, is called a line feed. The hexadecimal 0d is called a carriage return.
Pretty much all the programs on the Windows platform understand and expect the hexadecimal 0a0d pair in text.
The 0d0a pair of characters is the signal for the end of a line and beginning of another.

Almagest accounts for that where TDW may not have caught that based on what files he was working with as test files.
I've ran Wings files, Blender files, and 3d Max files through the basic code and all report proper out put.

Quote:

Originally Posted by Mister_M (Post 2786691)

So, things are nearly finished ? :D

Please, check my PM :03:

Quote:

Originally Posted by Jeff-Groves (Post 2786761)

Yes, exactly that!

I was looking for the right words, but I missed them. Sorry for my bad phrasing and thank you for correcting it.

Quote:

Originally Posted by Jeff-Groves (Post 2786761)

Almagest accounts for that where TDW may not have caught that based on what files he was working with as test files.
I've ran Wings files, Blender files, and 3d Max files through the basic code and all report proper out put.

:up:

One thing I am doing is reading each line as a string. Not each section of that string like one would to import the obj format.

I'm not changing that information so strings should stay as in the obj file.
All I'm doing is "Shuffling the cards" so to speak.

Now that does involve "Stacking the deck" in areas but it works for our needs.
:haha:

Quote:

Originally Posted by gap (Post 2786275)

Yesterday I have had an insight into a couple of files, and I definitely see some problems.

This is a quick comparison of the two files:

https://i.imgur.com/nRX8aua.png

Vertices (v)

As expected, 3D vertex coordinates are identical in number, order and value between the two files.

Texture vertices (vt)

For each file, there are more UV vertices than 3D vertices. This is expected because when you unwrap a closed surface for texturing it, you need to "cut" it in pieces so to make it as flat as possible and to avoid texture distortions. As a consequence texture vertices along the boundaries of "cropped" UV regions are duplicated.
There is a mismatch in number of texture vertices between the two files, the main object having a bigger number of UV coordinates than its UV2 copy. This is probably not good for TDW's importer, and it might be the consequence of me cutting further some UV regions when creating the main UV map from the AO one.

Vertex normals (vn)

Vertex normals are identical in number, order and value between the two files. Again, this was expected as the main and AO objects have the same 3D geometry.
There are more vertex normals than vertex coordinates. That should be the consequence of my models having hard edges in place. Since the vertices along those edges are considered as part of two or three surfaces (each pointing in its own direction) rather than one smooth surface, there is a need for the shared vertices to have more than one normal.
If I had removed hard edges before exporting the models, I am sure that the number of normals would have equated the number of vertices, and if I split the hard edges (as I did with the Blender exports), there would be as many vertices as the current number of normals.

Faces (f)

There is an equal number number of face definitions between the two files. Again, this is the direct consequence of the two objects being identical in their 3D geometry, thus having the same number of faces.
In a triangulated model each face definition is composed of three triplets of numbers, each triplet corresponding to one of the three vertices composing each face. Within each triplet, the first number should be the index to each of the 3D vertices connected to form a triangle; the central member of each triplet should be for texture vertices, and the third one for vertex normals.
- Between my two models, there is a partial mismatch in face order relative to the first (v) and third (vn) indices of each triplet. This mismatch might be caused by the fact that the main model has three materials assigned - one for the main texture, one for the railing and one for the rigging - whereas the AO model got only one material. Ungrouping materials in the AO model might have scrambled face order relative to the main model, since faces of the latter model are ordered on file depending on the material they use.
- There is a total mismatch of texture vertex indices, i.e. the second numbers in each triplet of face definitions. This mess might have been caused by the removal of "redundant" materials from the AO model (as above), but also by the cutting of some edges in the UV map of the main model (as explained before, there are more vt coordinates in the main model than there are in the AO model).

If Almagest could obviate to my mistakes, sorting out texture coordinates and faces in one click, that would be terrific. I realize that setting up the needed automated routines might be more complicated than it sounds though. Alternatively, I could try to manual edit my AO model in the hope to correct at least in part the problems above.
It that doesn't work, redoing the main UV mapping on the base of the AO model is still an option. That should not be a complicated task either. A little boring but not too complicated. For this particular task I think I would resort to Blender.
Blender's advantage is that, like Max, it supports multiple UV sets, and it got advanced and configurable shader setting. Very useful for previewing the combined effect of diffuse, specular, normal and AO maps. Unfortunately I am not as proficient in Blender as I am in Wings3D, but learning it better would be an investment for future projects.

Quote:

Originally Posted by gap (Post 2788535)

What I can tell you for sure is that if the main and the UV2 model are identical in all the respects except vt coordinate values, there is a 100% probability of success. With this idea in mind I have created an Excel spreadsheet which sorts vt coordinates for us. It is a rather crude tool, it requires a lot of manual data filtering, cutting and pasting; any mistake will cause wrong results, but if used correctly it does its job just fine.

If you are interested into testing it, I can upload it somewhere and post its link in the Almagest thread where it belongs more than here.

Couldn't be the problem coming from the fact that sub-objects (composing the final exported "diffuse object") are combined in a different order than in the "AO object" ?

If your excel sheet can solve the problem, let's try it. :up: Just explain to me how this works. :doh:

I know from reading the other thread that a question was raised about how the information on mapping is stored in a dat file.
A dat with only 1 map channel is stored in a simple block of data.
When We add say an AO channel? A TMAP section is added to the dat.
I'll have to refresh my memory on how the TMAPs actually work since it's been so long since it had to be done by hex editing. (Pre 2007!)
BUT! Only faces and texture coords are saved to the TMAP sections if I am recalling correctly.

I actually still have a program from 'back in the day' that exports and imports the different TMAP channels. So one could change or add just a TMAP and not have to re-import the complete obj like S3D does.

https://www.subsim.com/radioroom/pic...ictureid=12276

One thing I don't do is try to 'Guess' what went wrong.
Send me the files and I'll look at them.
I've probably forgotten more then most people know.
Not trying to be a smarmy butt head but I have been around the block more then once.

Quote:

Originally Posted by Jeff-Groves (Post 2788552)

One thing I don't do is try to 'Guess' what went wrong.
Send me the files and I'll look at them.

Here are .obj files which don't render correctly in game with the AO map : https://www.mediafire.com/file/ut1ss...O_map.zip/file

We are trying to understand why...

Quote:

Originally Posted by Mister_M (Post 2788544)

Couldn't be the problem coming from the fact that sub-objects (composing the final exported "diffuse object") are combined in a different order than in the "AO object" ?

Possible, but that's not the main suspect in my list of possible culprits.

What for sure can mess up the order of sub-objects (and faces) is material assignation. The main model and the AO model should have materials with similar naming (Wings3D sorts them in alphabetical order) assigned to corresponding faces. If, for example, a group of faces in the main model is given a material whose name is let's say 'wood', the equivalent faces in the AO/UV2 object should be given a material named 'wood_AO', 'wood_2', or something similar.

Quote:

Originally Posted by Mister_M (Post 2788544)

If your excel sheet can solve the problem, let's try it. :up: Just explain to me how this works. :doh:

Yes, it can. As I said, by using the output of my spreadsheet we can create an AO/UV2 obj file which is the exact copy of the main obj file, exception made for the values of texture coordinates.

Before I upload it, I need to be sure that it has enough rows for processing your model. Can you tell me the number of f and vt lines in the obj files you are processing? You can easily count them by pasting their content in an empty Excel document and by using the data filtering options or the COUNTIF function.

Quote:

Originally Posted by Jeff-Groves (Post 2788547)

I'll have to refresh my memory on how the TMAPs actually work since it's been so long since it had to be done by hex editing. (Pre 2007!)
BUT! Only faces and texture coords are saved to the TMAP sections if I am recalling correctly.

Thank you for your inputs Jeff!

Does that mean that faces and texture coordinates can be in a different order in the secondary map channel than in the primary one?

Quote:

Originally Posted by Jeff-Groves (Post 2788547)

I actually still have a program from 'back in the day' that exports and imports the different TMAP channels. So one could change or add just a TMAP and not have to re-import the complete obj like S3D does.

https://www.subsim.com/radioroom/pic...ictureid=12276

in future this tool might come in handy :up:

Well that was easy.
I compared the main obj and the AO obj in 010.
Your vertices do not match between the Main Hull and AO Hull.
(I suspected that but as I said. I'll not play a guessing game)
So anything else is going to be All screwed up on import!

What S3D does when importing the Main diffuse object and the AO object is this!

IF number counts are the same for the vertices? It ignores what those are in the AO object and only imports the faces and texture coords!

So basically S3D IS importing garbage you created.
That is the flaw AND the opening in S3D that I am working to fix!

Quote:

Originally Posted by gap (Post 2788563)

Can you tell me the number of f and vt lines in the obj files you are processing?

Hull1.obj :
2018 f
868 vt

Hull1-uv2.obj :
2018 f
2729 vt

It already seems problematic... :D

[QUOTE=Jeff-Groves;2788565]Well that was easy.
I compared the main obj and the AO obj in 010.
Your vertices do not match between the Main Hull and AO Hull.

Ah ! You replied while I was writting this...

Quote:

Originally Posted by Jeff-Groves (Post 2788565)

So basically S3D IS importing garbage you created.

Sorry, you mean : garbage that Wings3D created... :D

Quote:

Originally Posted by Jeff-Groves (Post 2788565)

That is the flaw AND the opening in S3D that I am working to fix!

You are trying to fix garbage created by Wings3D when exporting ?

Well. In the dat file format you CAN HAVE different counts on that information!
But the VERTS MUST BE EXACTLY THE SAME!

I'd suspect Wings re-ordered the Verts placement in the exported obj file thus faces and textures are screwed.

And You are the one that used Wings. It didn't to it by itself!
LOL!