J3D File Format

[Adzorn]

Is there some Documentation about the J3D file format?

I'm planning to build a J3D model Viewer for a work related Project (We use Dangerous Water Non Commercial Version at work) and i'm look for some documentation on the format, especially how to open it or decode it.

[Polak2]

We use Dangerous Water Non Commercial Version at work
a.And out of my curiosity what would that be?

b.There is no format documentation for J3D out there in the open and available to the public. There is 3dsmax plugin exporting (but no import) to J3D.

c. BTW would you be interested in deciphering compression of the .dem and other terrain related files residing in World folder?

[Adzorn]

a. There is a Non Commercial Version of Dangerous Waters Available. This Version contains some special feature e.g. DIS Protocol Support or a Build In database Editor.

http://www.minecraft-smp.de/images/Dw_NonCommercial.png

b. hmm.. there must be something around. Someone wrote the plugin and there are some tools to open the J3D format.

c. No, but if you require Height Maps, check out https://gdex.cr.usgs.gov/gdex/, you can find better and more detailed maps there (If you require DEM, then use MICRODEM to convert them)

[Aktungbby]

Adzorn!:Kaleun_Salute:

[Polak2]

There is a Non Commercial Version of Dangerous Waters Available. This Version contains some special feature e.g. DIS Protocol Support or a Build In database Editor.

Judging from the date (2004) this must have been some very early pre-release. Never heard of it as the earliest public release was I think in Dec 2005.

As far as plugin is concerned I am not sure who wrote it but AFAIK it came also from Sonalysts early in the game life. No documentation to my knowledge but I could be wrong.

[Polak2]

Here is some more information pertaining to J3D:

There is program reading and viewing the format. It is called 3D Object Converter and it can be found here:
http://3doc.i3dconverter.com/
It can only read the format and you need to register it to be able to save to other formats (but not J3D).

To be able to save to J3D one needs plugin which works only with old 3dsMax3.1 and in its original form has pretty low vertex limitation for models (can be hacked I think).

[Adzorn]

No, its not a Pre-Release, its the normal Version of Dw but with some Special Features for Military Use.
The main feature is the ability to use the DIS-Protocol (Distributed Interactive Simulation). With this protocol its possible to connect this Version with outer Simulator like Prepar 3D (https://www.prepar3d.com/) or VBS3 (https://bisimulations.com/products/virtual-battlespace).
Also a build in Database Editor is included in the Version.

I know the 3D Object Converter. I contacted the programmer behind but he was not willing to share his knowledge ^^

[Polak2]

I contacted the programmer behind but he was not willing to share his knowledge
Well I shared with you all we know here about J3D.
About those DEMs in World folder they must be some proprietary formats ... or compressed. Regular DEM viewers fail to open them.
Oh well. :hmmm:

[Battlewagon3]

I know that it's irritating, but I'll probably have to post this on some other threads as well:

1. The models from LWAMI, when installed according to the instructions, are invisible in my installation of DW.

2. When I have tried to install them manually using GRP (by Adam Milazzo), I had the same problem.

3. In addition, when new models (with or without textures) are installed using GRP, the .dds textures on other subs disappear, which means that my 3d.grp and 3d.ndx files have to be replaced in order to get DW working again. Replacing the .dds textures with .bmp textures doesn't work, either for restoring the textures, or getting the models working.

3. DWEdit is great for modding existing content (if you leave the graphics alone), but the Import and Export features don't appear to work, at least on my computer.

4. Due to the above problems, repaints are also not possible on my computer.

Does anyone know if I need to install anything else, or change any settings, so that these mods will work? Thank you.

[Aktungbby]

Battlewagon3!:Kaleun_Salute:

[Battlewagon3]

UPDATE: I've managed to get custom models working, but I'm still having trouble with default textures disappearing.


Does anyone else have anything to add?

[Jeff-Groves]

I can provide enough info on the format to allow you to write a viewer/exporter.
I've only looked at it for a few hours so still figuring out the last bit of unknowns. One could write a new exporter for 3D Max or other programs once I figure it all out.
:hmmm:

Thank you TLAM Strike for a few models I downloaded to look at the structure.
Those were a great source to work with.

[Jeff-Groves]

Looking at the P4_Midget_Sub j3d file header in SweetScape 010 We have this:
EB 0F 00 00 EC 0E 00 00 02 00 00 00 12 00 00 00
10 00 00 00

EB 0F 00 00 as an integer is 4075. That is the number of verts in the j3d file.

EC 0E 00 00 as an integer is 3820. That is the number of polys in the j3d file.

02 00 00 as a short is 2. That is the number of textures used by the j3d file.

12 00 00 00 as a short is 18. That is the number of 3d models in the j3d file.

10 00 00 00 as a short is 16. That is the number of texture mappings in the j3d file.

So we have 4075 verts, 3820 polys, 2 textures, 18 3d models, and 16 texture mappings.

The next stuff is all the verts and texture co-ords. I'll break that down in the next post as there is some info I am not sure of at this time.

[Jeff-Groves]

Following the header I described is a large block of data which includes the vets and faces of the 3D objects. Let's look at a chunk of that.

66 4C 41 BC BB D2 66 BF 6A AA 68 C1 00 00 00 00
55 55 AD AE 40 CA 2B 38 00 00 80 BF 00 00 00 00
00 00 00 00 16 F5 09 3F

This chunk gives us the verts, an unknown I suspect may be tangents, and our texture co-ords. We read them all as floats.

66 4C 41 BC BB D2 66 BF 6A AA 68 C1 = v 0.011798 -0.901653 -14.541605

00 00 00 00 = seems always the same. maybe a divider?

55 55 AD AE 40 CA 2B 38 00 00 80 BF = unknown. Maybe tangents?

00 00 00 00 00 00 00 00 16 F5 09 3F = tex co-ords. 0.000000 0.000000 0.538996
This one is interesting as so far as I have found the 3rd co-ord of 0.538996 must be subtracted from 1 to get the proper number.
Then moved to the secondary position on export! vt 0.000000 0.461104 0.000000

If someone could send me a couple of the Games original 3D files that may help.

[Jeff-Groves]

A template and script to work with j3d files is being created.
However. I am creating a special file to work with those that will require you to buy a legal copy of SweetScapes 010 Hex Editor.

I put a lot of time in to help with stuff. SweetScape has the best hex editor out there!
I support them and they add nearly anything requested. Spend a few bucks to support a great bit of software! 010 is the best tool a modder can own!

[Jeff-Groves]

Now. After all the above stuff we come to the faces.
It's all pretty boring to me but you all want the information.
What happens with the j3d file is ALL verts and tex info is one large block with no separation of models. YET!
Now we look at the faces needed. Those also run as one large block with separation coming later in the file.

[Jeff-Groves]

Again looking at the P4_Midget_Sub in 010 I jump to the faces section.
I get the jump address by multiplying the number of verts (4075) by 40 (which is the number of bytes in the chunk which gives us the verts and tex co-ords stuff) then add 20 for the header block.

I work with the 3DS file provided and export it from 3DS Max as an obj file.
3D Converter will also export as an obj if you have a registered version.
The trial will drop every 5th vert info. $50 USD for lifetime upgrades is a deal. And yes I have a registered version.

[Jeff-Groves]

One thing that will make it easier to read and match the j3d file info to the exported obj file is to rotate the model on export.

Now the faces info.

If you jumped to position 163020 decimal your at the start of the faces info.
You will see:
01 00 02 00 00 00 02 00 03 00 00 00 03 00 04 00
00 00 04 00 05 00 00 00 05 00 06 00 00 00 06 00
Blah blah blah.

ADD 1 to each Short. So 01 00 should be converted to 2 for export.
02 00 would be 3, 03 00 would be 4, and on and on.

The current export options accumulate all the faces info! You will hit the
01 00 02 00 00 00 02 00 03 00 00 00 03 00 04 00
00 00 04 00 05 00 00 00 05 00 06 00 00 00 06 00
Blah blah blah.
stuff 5 times in the example sub I am working with.
I suspect that is just objects imported during the building of the unit.
I'll check other files to see if this holds true.

So read the 01 00 02 00 00 00 as such for faces:
2/2 3/3 1/1

Current export options will give you:
1/1 2/2 3/3 and so on.

[Jeff-Groves]

Working through the faces we come to some good stuff!
How to read all the info into a workable shape.
3D object converter is fantastic for giving me a lot of information here.
As a side note? I'm checking with a few old friends to see if I can get someone to write a new j3d exporter.
Looking at the files closer? I may install my old VC++ program and do a universal converter that reads the obj and mtl file to convert.
It is a pretty simple file structure. Almost a joke compared to Gr2 or even the dats used in SH Games.

[Jeff-Groves]

Bought and installed DW today so I can check more files.
Also got the decompress program in the downloads section.
I just stopped by home on a return trip from Connecticut for a few hours as I load gear for a job in Nashville. So I grabbed my VC++ stuff. I needed that anyway for other projects.

[Jeff-Groves]

Here's the start of the 010 template for j3d files.
http://www.mediafire.com/file/f93unec7tic2ubz/j3d.7z/file

This template will take you to the start of the faces.
It reads the file just as is!

[FPSchazly]

Cool stuff. Keep it up! :Kaleun_Cheers:

[Jeff-Groves]

Thank You!

I plan to do lots more in the long run.
A stand alone exporter is pretty simple to do but I'd like to create a new program to create new j3d files.

[Polak2]

Very interesting!

It could be perhaps handy for you to know that 3DObject converter handles reading j3d and displaying it including UWV coordinates materials etc. for nice and easy conversion to various editable formats.

What is still problematic is :
a. plugin to go back from any editable format back to j3d is only available to 3dsMax3.1 which unfortunately is not so popular or easily available any longer
b. what is even worse in that plugin there is a limitation of vertices of 32768 which is rather annoying and prevents most of the more complex models from exporting.

Do you think such a limitation could be removed or better yet new plugin for newer 3dsMax versions could be made ?

[Jeff-Groves]

I have 3DConverter. A legal lifetime version.
I also have 3D Studio 3.1 and the j3d exporter.

Now is the exporter the limiting factor on verts? Or the Game engine?

I can Hex edit a file to run way over that but would need to test it in Game.
I bought DW from Amazon but it won't run.

[Polak2]

Now is the exporter the limiting factor on verts? Or the Game engine?
Exporter, 99.9% sure. RA modders having j3d models which are way beyond this limit of 32768 . If you can hexedit this barier I can gladly test it.

[Jeff-Groves]

Kara2 seems to be the largest 3D file and does not exceed 32181 polys
nor more then 29009 verts.
Of which only 22750 verts are shown in 3D object converter.

Interesting to note you need to jump down the file to decode the verts.

The 212_esm. j3d file is even more interesting.

[Polak2]

Kara2 seems to be the largest 3D file and does not exceed 32181 polys
nor more then 29009 verts.
Of which only 22750 verts are shown in 3D object converter.
Indeed, I concur. I am little rusty on this j3d quest, so I thought that they managed to "sneak" larger file.

What is interesting 212_esm. j3d an ESM mast? I assume the interesting part is in hex look of the file.

[Jeff-Groves]

What is interesting 212_esm. j3d an ESM mast? I assume the interesting part is in hex look of the file.

Yes. The header part which lays out the number of verts and such is different. Still looking at it to try to figure out what the extra area is for.
I can calculate 283 verts of which 163 show up in 3d Object converter.
The extra space is what I don't understand at this time.
It's the first 4 bytes of the file.

EDIT: The first 4 bytes appear to mark the file as having an animationable part? That would explain the missing verts in 3D object converter also. Several files from DW stock have the exact same 4 bytes. Windsock is one.

I'll add a check to the template for 010 to jump the first 4 bytes if it is found. That will read all verts properly and get us to the faces.

[Polak2]

You are possibly right animation perhaps ....If my memory serves me right that ESM antenna (tiny bit) extends from the thicker mast above the surface of the water.

[Jeff-Groves]

So far doing a check does jump the header and get me right to the faces section.
Another odd file is the airdepthcharge.j3d
3D Object converter does not like that one.
:haha:

[Polak2]

Another thing is that in the example of type 212 which we have started this very intriguing conversation, animation parts like 212 periscope 1,2 , snorkel, radar or mentioned ecm antenna are not part of the main 212.j3d model . The sail is empty inside. Those parts seem to be called once the appropriate command is issued.
I believe this happens throughout all other models.

[Polak2]

Another odd file is the airdepthcharge.j3d
3D Object converter does not like that one.
Not that this is of most importance, but I think that airdepthcharge.j3d is not used in the game. DWEdit has some nifty feature linking entities to their j3d models. Air deptcharge according to DwEdit uses depth.j3d and depth3.j3d which looks like the same models to me.

[Jeff-Groves]

As I get the time I'll write a script to export the info from the j3d files as is.
3D Object Converter just does not export all the info and I'd like to know the reason behind that.

I may wipe the old NoteBook and re-install Win 95 or so.
Then I could run the old 3D Studio and j3d exporter to see what the export options are.

[Polak2]

3D Object Converter just does not export all the info and I'd like to know the reason behind that.

I think 3DObject Converter takes all animated parts of surface units (usually guns, radars, torps) and bunches them to one single coordinate ....usually inside the hull so unless one switches to wireframe mode, one may never know that they are there...lol.

[Jeff-Groves]

I think 3DObject Converter takes all animated parts of surface units (usually guns, radars, torps) and bunches them to one single coordinate ....usually inside the hull so unless one switches to wireframe mode, one may never know that they are there...lol.

3D Object Converter seems to be skipping some verts in some cases.
Or maybe just optimizing them and condensing the files.
That's fine for what it does but does not help when looking at a possible new way to convert to the j3d format.

And I have seen the way things may be out of place in the files.
It's much like SH3/4 where the location connection to parent location is ignored by every exporter ever written.

That's due to the way any given exporter is coded.
If a part is assigned as a child of a part at say 1, 5, 9 in the X, Y, Z? Then it inherits that position in the Games.
IF that is not coded into the exporters? You get the true X Y Z but not the inherited position so the weird placement thing.

[Battlewagon3]

Here's the start of the 010 template for j3d files.
http://www.mediafire.com/file/f93unec7tic2ubz/j3d.7z/file

This template will take you to the start of the faces.
It reads the file just as is!


I'm replying to thank you now before I forget. It could be a few months before I chase this down properly. I've been very busy learning how to code (really) for my studies, so games modding has taken a backseat. As you'll see from my other posts, though, every few months I come back with questions and try a few more things. I'm not ignoring you!

[HoneyFox]

I took several hours researching the j3d file format, and here is what I've found (see attachment).

It's written in C++ style, although C++ doesn't support declaring array of uncertain length, but... you get what I mean.

PS1: There are still some unknown items, but they seem to be constant during my research over dozens of .j3d files.
PS2: It's still far from having a new exporter. Major challenge is to find a suitable input model format and write a parser for it, especially when dealing with models containing hierarchy.
PS3: Actually I'm not sure if that use_tex and render_order is correct or not, I found no value to store the material index of model, but it doesn't seem to match if we assume it is.

[Polak2]

Interesting work and progress.

Are you trying to make a new exporter plugin like DWj3dexporter2004 which has some limitations of polygons being able to export as well as being compatible with a very old version of 3dsMax?
I assume that you are aware that j3d is part of the 3D ObjectConverter and can load models but not save them back to j3d.

Perhaps there is something we can help with?

[Polak2]

Perhaps of some help would be a structure of DW database where some of the j3d elements are mentioned.

I think it is included in DWEdit1_7_4_1 which was in DW Download section on this forum.

[HoneyFox]

Perhaps of some help would be a structure of DW database where some of the j3d elements are mentioned.

I think it is included in DWEdit1_7_4_1 which was in DW Download section on this forum.

Oh, my aim is to have a tool for exporting j3d models for JFC. I don't know if this file format has changed or not when it comes to DW.:o

Anyway this is the current workflow: modeling in Blender 3.x -> Blender 3.x Wavefront obj exporter (I modified the wavefront obj exporter so that the hierarchy/offset info can be exported as well) -> Wavefront obj file -> My exporter -> J3D file -> cmpUtils.exe -> GraphicsDB.

Too bad that subsim doesn't allow me to upload .zip file attachment...

[Polak2]

Oh, my aim is to have a tool for exporting j3d models for JFC. I don't know if this file format has changed or not when it comes to DW.:o


Word of caution: as DW j3d plugin had a limit of 32K polys on the models it is very probable that JFC (being the earliest) would have some hard limits on 3D models as well. It was dictated by the HW in those days.

[HoneyFox]

Word of caution: as DW j3d plugin had a limit of 32K polys on the models it is very probable that JFC (being the earliest) would have some hard limits on 3D models as well. It was dictated by the HW in those days.

It's expected since the data structure mentioned in J3DFileFormat.h above has lots of items using “short"/"unsigned short” type, which implies that there will be a limit of 32768(if it's signed) or 65536(if it's unsigned) for both vertices and triangle faces. Similarly there is a limit of up to 128 textures and 256 or 128 materials since it's using "char" type, and up to 32 characters for names of objects in the model.

Luckily for JFC we don't really need that high poly count in most cases. DW's graphics quality is definitely in another level so exceeding-the-limit becomes more often.

[Polak2]

There are not many differences between graphics (3D model-wise)for JFC and DW.

https://i.ibb.co/cJFZBFH/3dconverter-u-LBRpn-Gu30.png (https://ibb.co/rk6zj6S)

In fact, hardly any.

Which begs the question: are the JFC mission playable on DW? The interface of JFC is a little dated and so is the DW but not as much.

[HoneyFox]

As someone requested, I've pushed my tool to github:
https://github.com/HoneyFox/J3DFileConverter

Feel free to fork/create merge requests. :cool:

[faxnico]

Thank you for sharing.

Just a couple of points...

The value at Model+0x58 is a dword, and if it equals 1, the model is drawn through DrawPrimitiveUPs. This occurs once or twice during the recursive traversal of the hierarchy.

As for Model+0x5C, it represents another dword, typically set to 0, and points to an item within an array of render data. Each structure in this array is 0x5c bytes in size and contains at least two user-provided vertex data buffers for the DrawPrimitiveUP calls mentioned earlier, filtered by the condition Model+0x58. The second buffer is utilized, and the draw call is executed, only if Model+0x294 is true as well. This draw call is wrapped by special render state enable/disable code...

Hope this helps.
Keep up the good work.

[Polak2]

Let me be the 1st to congratulate you on this milestone achievement.

Not sure if Dangerous Waters also uses the same J3D format or not, I haven't tested it with DW.
Yes, it does. In fact, as my earlier picture shows the models in JFC are nearly identical with SC (Sub Command) and subsequently DW (Dangerous Waters).
It will be interesting (time permitting) to test the Obj models export to J3D for use in all of the above Sonalyst titles.
:Kaleun_Thumbs_Up: and :Kaleun_Applaud:

[Jeff-Groves]

Nice work HoneyFox!
I lost interest with the format way back. Still have the 010 scripts and such.
I'll check your work and see what I missed. Always good to learn from others.
:salute:

[HoneyFox]

The value at Model+0x58 is a dword, and if it equals 1, the model is drawn through DrawPrimitiveUPs. This occurs once or twice during the recursive traversal of the hierarchy.

As for Model+0x5C, it represents another dword, typically set to 0, and points to an item within an array of render data. Each structure in this array is 0x5c bytes in size and contains at least two user-provided vertex data buffers for the DrawPrimitiveUP calls mentioned earlier, filtered by the condition Model+0x58. The second buffer is utilized, and the draw call is executed, only if Model+0x294 is true as well. This draw call is wrapped by special render state enable/disable code...


I'm curious about how you get these info. Maybe via debugging JFC or debugging DW?
For "Model+0x294", I don't quite get it. AFAIK, the "Model" size is 0x60(96) bytes. Model+0x294 seems to be way out of bound, unless it is some runtime-only structure different from J3D file structure, or it is some optional field of "Model" which only becomes available when the "Model+0x5c" is not 0.

[faxnico]

Debugging and offline/static analysis.
I had this sorted out a while ago to code a J3D loader replacement with something more modern (GLTF).
+0x294 was an offset to the runtime structure indeed (S3DModel).
The model/mesh/subset data size is 0x60 bytes.
The render state change that wrapped that 2nd drawprimup was, on the other hand:
SetRenderState(D3DRS_ZFUNC, D3DCMP_NEVER);
DPUP();
SetRenderState(D3DRS_ZFUNC, D3DCMP_LESS);
SetRenderState(D3DRS_ZENABLE, TRUE);
Maybe for some decals rendering, I don't know.
:salute:

[Aktungbby]

faxnico!:Kaleun_Salute:...after 14 years!??:yeah:

[HoneyFox]

Debugging and offline/static analysis.
I had this sorted out a while ago to code a J3D loader replacement with something more modern (GLTF).
+0x294 was an offset to the runtime structure indeed (S3DModel).
The model/mesh/subset data size is 0x60 bytes.
The render state change that wrapped that 2nd drawprimup was, on the other hand:
SetRenderState(D3DRS_ZFUNC, D3DCMP_NEVER);
DPUP();
SetRenderState(D3DRS_ZFUNC, D3DCMP_LESS);
SetRenderState(D3DRS_ZENABLE, TRUE);
Maybe for some decals rendering, I don't know.
:salute:

This additional 2nd DPUP is rendered with ZFunc = Never? If ZTest is enabled previously, nothing will be drawn then. :o That's quite strange, even for decal/VFX/... transparent mesh rendering (They usually have ZENABLE = True, ZFunc = Less/LEqual and ZWRITEENABLE = False). Does it instead have any stencil-test related API call nearby?