Exporting 3D models from GR2

[skwasjer]

Originally Posted by Anvart

Everyone goes by own way
*********************
You speak about *.dat & S3D?
Look example...
File - Room_CR.GR2
Using Privateer's GR2_Exporter.exe - 55 objects (*.obj) have been exported/created... during 3 minutes (on my not fast machine ).
Two objects have very big size of obj-file...
...

CR_BSide.obj - file size = 407 223 KB



CR_FSide_Stairs.obj - file size = 657 264 KB



May be, these objects can be divided into smaller with (on) corresponding single-materials...?
...
For those who has not understood...
For comparison: SH4, file NSS_Porpoise_CR.dat, exported object Tzevi01_03_LM2.obj - file-size 2 658 KB...
Whether S3D can process such big file... 657 264 KB...? simple experiment has shown cannot...



... and pressing of "Continue" button leads to S3D hanging.

...
and Look forward - multi-materials...

Ok since you brought this up, I took a look, and I'm sorry to say that the OBJ-files exported are highly unoptimized. For every subobject, the entire vertex, normal and texture array is exported as a whole, each time, over and over.

The first subobject (on material) is:
- 7517 vertices
- 7517 vertex normals
- 7017 texture coordinates
- 5496 faces

In privateers export however it is:
- 351977 vertices
- 351977 vertex normals
- 351977 texture coordinates
- 5496 faces

What?? Why export a massive array of which you only use 2% of data?

For the next subobject, he exports the same vertices, the same texture coordinates, the same normals, only the face-assignment obviously differ... That's roughly 30MB of extra data, for each next subobject, data you don't need because it's already written once in the file. I don't want to stomp on privateer's work, but not reusing the vertex, normal and texture arrays is bad coding. No wonder the file is so big. It should be not much more than 1/9th that size for this entire OBJ-file (and that's just an estimate, it might be even smaller).

Here's an image:



To import this I cut off all subobjects trailing the first. The resulting OBJ-file was 30 MB. After import I got the above. Exporting it again from S3D resulted in an OBJ-file of 856KB !! That's 97% less bytes.

Then there's the subject of S3D failing on this file reporting 350K vertices, but that makes sense since DAT-files can't hold models larger than 65K vertex indices and 65K-texture indices. Splitting the object up removes the problem as an object of 7517 vertices as per example above clearly is not a problem for S3D/DAT.

Anyway, I've looked into GR2 files myself and have written code for it in april too (to analyze), so it was an honest suggestion that it can be done and is much easier. It doesn't matter in the end, it's privateer's project, and I appreciate his work (glad he does so), and if he continues on the OBJ-path I hope he can improve it further (also based on the above analysis) or perhaps he already knows. In the end it does not matter, even if he does not improve on the above, you can break the model up in a 3D studio, or by hand in notepad like I did, and after import in S3D it will drop any unused data (Pack3D won't).

Regards, skwas

Nice post skwasjer,


Yes I am aware of the exporter doing that.
It's the old code I'm building off of that's looping garbage.
It will get corrected in a later release.
That's why I said this is only a teaser release.

I'm not a super programmer by any means.
I'm more a stumble through it and get lucky programmer.

[skwasjer]

I didn't mean to put you down on the coding end, but as Anvart posted this info of an OBJ file of 600MB and S3D throwing an (obvious) error I scratched my head and had to take a look, because that big didnt make any sense. We'd be dealing with models of 50 to 100 million vertices.

You're doing ok sofar.

I didn't take it that way at all Mate.


I did fix the bad loop and that 600 meg file comes out at
just over 51 megs now.

If I dropped the normals it would be smaller.
Around 38+ megs.

[Anvart]

Originally Posted by skwasjer

I didn't mean to put you down on the coding end, but as Anvart posted this info of an OBJ file of 600MB and S3D throwing an (obvious) error I scratched my head and had to take a look, because that big didnt make any sense. We'd be dealing with models of 50 to 100 million vertices.

You're doing ok sofar.

Thanks for the situation analysis, Skwasjer.
At last i have seen the constructive answer...
I suspected not optimised creation of obj-file too... but i wanted to see your answer.
... or you have not understood my intentions...? with my english i cannot be assured...
...
and Granny Viewer's info about...

[The General]

I can't understand a word you guys are talking about in this thread, but it looks important . Am I to understand that you guys are working on 'Importing' ship models from previous Silent Hunter's into SH5?

[skwasjer]

Originally Posted by privateer

I didn't take it that way at all Mate.


I did fix the bad loop and that 600 meg file comes out at
just over 51 megs now.

If I dropped the normals it would be smaller.
Around 38+ megs.

That's great

Originally Posted by Anvart

Thanks for the situation analysis, Skwasjer.
At last i have seen the constructive answer...
I suspected not optimised creation of obj-file too (for this purpose i have written #65)... but i wanted to see your answer.
... or you have not understood my intentions...? with my english i cannot be assured...

I just thought 600 MB for an OBJ-file was crazy (and total export > 1GB from a single 50 MB GR2-file)! So I had to look why... no ill intentions.

[skwasjer]

Originally Posted by The General

I can't understand a word you guys are talking about in this thread, but it looks important . Am I to understand that you guys are working on 'Importing' ship models from previous Silent Hunter's into SH5?

No, it's an attempt to work around GR2, export its assets and reuse them in DAT-files for SH5. What we don't know at this point if it is actually possible to replace GR2 completely (especially regarding animations and more complex models like interior that may have hardcoded stuff vs GR2), but who knows. It's a step in the right direction anyway. To import previous SH-units, that should technically already be possible from what I've read, although there are/were (?) issues with AO-maps. In the end, the conclusion may be drawn that we 'do' need GR2, and then you will see attempts to write GR2 importers for sure

[Anvart]

Originally Posted by skwasjer

...
to look why... no ill intentions.

Why you have supposed it?
...
My EN is difficultly understood (or not is understood completely)...?

Dropping the loop puts all objects in one mesh again.
But I'm looking at a function to make the loop optional,
to break the objects up, then filter out the unused verts.
The VGC library is what I'm looking at to do the filtering.

I've found no good way yet to do it differently.
There's no pointer I have found to break them up without the extra garbage.

[skwasjer]

You do not need to filter the vertex/uv/normal list yourself, just drop it in the first section of the file like you already have, and then you can break up the file in subobjects with an g-token... Then you can use the same vertex/uv/normal list, still all in one big file. This is equaly valid as putting shorter lists of vertex/uv/normals before each face list (although the latter would be a little better to read visually).

v x,y,z
..
..

vn x,y,z
..
..

vt u,v(,w)
..
..

g object1
(usemtl)
f vi/vti/vni etc...
..
..

g object2
(usemtl)
f vi/vti/vni etc...
..
..

I understand the 'g object' command.
What I have not found is a way to break the list down in the GR2 files.
They are packed as one long list.
(That's why the loop)

I've not found a pointer as to where to break this list.

[skwasjer]

You don't have to is what I am saying. You just write them all out only once at the top of the file, and then loop all subobjects and write them out. That should do, afaik. Did you already try this, and/or does this give specific problems?

[skwasjer]

Or do you mean there are no subobjects in that one big mesh? In that case you could do a split on material (I think Anvart suggested this too). This is also what I did in my example above.

I can split the Tri-Groups as there are pointers as to where to break them.
The materials are then linked to those.
But I've found no link to the Verts, Normals, Textures listing.
It's one long chain.

So no sub objects.
If it's in the Extended Data?
I have not spotted it yet.

I've also not extracted any data from the extended section yet.
It's not placed in the GR2 like everything else.


Now I have to go to my real job.


That gives me all day to ponder on things.