MaHuJa
03-25-06, 09:47 PM
I've been thinking about it ever since I (a couple hours ago:-j) read barlymans post in the suggestions thread, and thought about the things I put in the reply below.
I will, however, need to have someone else writing the graphical interface... I'm thinking that one can make the gui totally pluggable. (I'll be using C++, but if your programming language of choice can use C interfaces, and I believe most suitable for the task can, we can make it work.)
Essentially, if I were to make a replay viewer, it would likely have the following features when "finished":
-The DDI from the navmap, with information about depth, speed, etc. would be available.*
-Creating "keyframes" and indexes, for fast forward/backward motion. I'm intending to cache it in a file right besides the replay file as an option.
-Exporting the state of the replay into a mission file, so that, using the same interface, you can see (approximately, due to TA/dipper length/depth etc) what the player could see at that point
-The possibility of toggling the three categories of contacts (truth, link, own solutions) between fully visible, transparent, and invisible. (slider?)*
-If a gui-maker decides to go advanced enough, you could perhaps get the 3d view too...*
-We'll have to see what more ideas pop up.
* Depends on whoever makes the gui in use, I guess...
I'm supposing that SCS will give me the specs of the replay files, etc. - I hear they have been similarly friendly before, with database editors.
As you might guess from the poll, I'm gauging interest. Would be a bummer to spend the time on it if noone was going to use it anyway.
--- --- --- --- --- --- ---
Barleyman, I read through your post in the suggestion box, but found that this big reply doesn't quite belong there, so I made a new thread instead.
I do not think you see quite the depth of the problems... I'll try to outline them for you.
First of all, I agree that replays as they are now are unacceptably slow when moving backward or forwards in time. However, your thinking haven't brought us much closer to a solution at all, because you overlook too many things.
Secondly, some of the idea was to (already) save data so that a replayer could show the whole thing in the navmap or similar interface, complete with 3d. Only, such a replayer hasn't been made yet. (I believe one could even compute the sensor data etc along the way, just so that is said. You could essentially see everything the user playing it saw.)
When creating a system like the replay saver, you have to balance a few factors:
-File size
-Speed
-Accuracy
-Continuity effects (the ability to move forwards and backwards in the replay - which is broken now)
(Continuity is a bad choice of word, but I couldn't think of anything better.)
In general, anything you do to improve one, will negatively impact at least one of the others.
In addition, you have some additional restrictions on what your system must be able to handle:
-Variable number of platforms (torpedoes launched, aircraft shot down, ships sunk then removed)
-Difference direction front vs travel should be accurately represented.
-You need to interleave data about contacts made, and their state at the time. Remember these go from bearing lines updated every few seconds to solutions.
For example, they could store the positions in floats rather than doubles (32 and 64 bit floating point numbers) to reduce size, but at the cost of accuracy.
With the game engine using double-precision floating points natively for positions, probably with good reasons, it's only natural to use these 8-byte values throughout.
They could send it through a compressor to reduce the file size, with only a very slight hit on speed, but this would very likely be very bad for continuity, since it would have to be decompressed start-to-end.
AFAIK (if I remember the SCS statements correctly) they do leave out some data, primarily positions, rather "adding it up" as they process it from start to end. Reduction to filesize, bad for continuity and speed.
They could add data to the replay file only when something changed (and otherwise the reader would just calculate from last information). Speed hit (for the reader), continuity hit, negligible accuracy hit. (And this presumes waves, wind gusts etc. conform to chaos theory predictability)
4byte X latitude, 4byte Y latitude, 2 byte altitude(for replay purposes not really needed), 2 byte direction and 2byte speed recorded once a second
This lacks way too much metadata to be of any real use.
Look at this example:
struct objectState {
short id; //2 - need to know what object we're dealing with. Corresponds to the number in truth mode.
// position - assuming z to be altitude here
double px; //8
double py; //8
float pz; //4 - they would get away with float if size was a concern, but at some speed cost.
// Where the object is pointing, direction
float dx; // 4 - pitch
float dy; // 4 - course
float dz; // 4 - roll
// And that is if they aren't using quaternions, which is often a better idea than euler directions, but is four floats.
// How the object is moving
float vx,vy,vz; // 12
// This is because the direction of flight/movement is *not* always equal to where the object is pointing. This is especially because of wind and water currents, but not only. Check with your average flight simulator that shows where the nose is pointing and the actual flight direction
}
That's 66 bytes (presuming I didn't think of and add yet another entry since I summed it). Presuming one of those a second, one object will create nearly 250kB per hour. That means four objects, for one hour, will be 1MB. 100 objects? 25Mb. I think this is about the rate DW does produce (not sure), but DW does it more often, since once a second will have aircraft maneuvering jumping about like some laggy flight sim. (Ever played flight sims, and tried to chase down a guy lagging very badly? With his aircraft "blinking" around madly?)
And in addition to that, there would have to be a way to "package" that, and keep tabs on the "when" for each piece of data (without it, you have to assume a constant number of entries for objects... see "additional restrictions above"), and add the contacts etc, which will lead to additional overhead. Look to the mpeg, avi, ogg, and matroska (mkv, mka) multiplexer formats to see what I mean. (Though I guess one could package those features into the entries themselves, it would likely become far less efficient in all ways.)
The most "backward compatible" would be to copy the video codec trick of "key frames". Every five minutes or so, write down the whole state of everything that is in there right then. Then the reader will only have to move on from there.
I will, however, need to have someone else writing the graphical interface... I'm thinking that one can make the gui totally pluggable. (I'll be using C++, but if your programming language of choice can use C interfaces, and I believe most suitable for the task can, we can make it work.)
Essentially, if I were to make a replay viewer, it would likely have the following features when "finished":
-The DDI from the navmap, with information about depth, speed, etc. would be available.*
-Creating "keyframes" and indexes, for fast forward/backward motion. I'm intending to cache it in a file right besides the replay file as an option.
-Exporting the state of the replay into a mission file, so that, using the same interface, you can see (approximately, due to TA/dipper length/depth etc) what the player could see at that point
-The possibility of toggling the three categories of contacts (truth, link, own solutions) between fully visible, transparent, and invisible. (slider?)*
-If a gui-maker decides to go advanced enough, you could perhaps get the 3d view too...*
-We'll have to see what more ideas pop up.
* Depends on whoever makes the gui in use, I guess...
I'm supposing that SCS will give me the specs of the replay files, etc. - I hear they have been similarly friendly before, with database editors.
As you might guess from the poll, I'm gauging interest. Would be a bummer to spend the time on it if noone was going to use it anyway.
--- --- --- --- --- --- ---
Barleyman, I read through your post in the suggestion box, but found that this big reply doesn't quite belong there, so I made a new thread instead.
I do not think you see quite the depth of the problems... I'll try to outline them for you.
First of all, I agree that replays as they are now are unacceptably slow when moving backward or forwards in time. However, your thinking haven't brought us much closer to a solution at all, because you overlook too many things.
Secondly, some of the idea was to (already) save data so that a replayer could show the whole thing in the navmap or similar interface, complete with 3d. Only, such a replayer hasn't been made yet. (I believe one could even compute the sensor data etc along the way, just so that is said. You could essentially see everything the user playing it saw.)
When creating a system like the replay saver, you have to balance a few factors:
-File size
-Speed
-Accuracy
-Continuity effects (the ability to move forwards and backwards in the replay - which is broken now)
(Continuity is a bad choice of word, but I couldn't think of anything better.)
In general, anything you do to improve one, will negatively impact at least one of the others.
In addition, you have some additional restrictions on what your system must be able to handle:
-Variable number of platforms (torpedoes launched, aircraft shot down, ships sunk then removed)
-Difference direction front vs travel should be accurately represented.
-You need to interleave data about contacts made, and their state at the time. Remember these go from bearing lines updated every few seconds to solutions.
For example, they could store the positions in floats rather than doubles (32 and 64 bit floating point numbers) to reduce size, but at the cost of accuracy.
With the game engine using double-precision floating points natively for positions, probably with good reasons, it's only natural to use these 8-byte values throughout.
They could send it through a compressor to reduce the file size, with only a very slight hit on speed, but this would very likely be very bad for continuity, since it would have to be decompressed start-to-end.
AFAIK (if I remember the SCS statements correctly) they do leave out some data, primarily positions, rather "adding it up" as they process it from start to end. Reduction to filesize, bad for continuity and speed.
They could add data to the replay file only when something changed (and otherwise the reader would just calculate from last information). Speed hit (for the reader), continuity hit, negligible accuracy hit. (And this presumes waves, wind gusts etc. conform to chaos theory predictability)
4byte X latitude, 4byte Y latitude, 2 byte altitude(for replay purposes not really needed), 2 byte direction and 2byte speed recorded once a second
This lacks way too much metadata to be of any real use.
Look at this example:
struct objectState {
short id; //2 - need to know what object we're dealing with. Corresponds to the number in truth mode.
// position - assuming z to be altitude here
double px; //8
double py; //8
float pz; //4 - they would get away with float if size was a concern, but at some speed cost.
// Where the object is pointing, direction
float dx; // 4 - pitch
float dy; // 4 - course
float dz; // 4 - roll
// And that is if they aren't using quaternions, which is often a better idea than euler directions, but is four floats.
// How the object is moving
float vx,vy,vz; // 12
// This is because the direction of flight/movement is *not* always equal to where the object is pointing. This is especially because of wind and water currents, but not only. Check with your average flight simulator that shows where the nose is pointing and the actual flight direction
}
That's 66 bytes (presuming I didn't think of and add yet another entry since I summed it). Presuming one of those a second, one object will create nearly 250kB per hour. That means four objects, for one hour, will be 1MB. 100 objects? 25Mb. I think this is about the rate DW does produce (not sure), but DW does it more often, since once a second will have aircraft maneuvering jumping about like some laggy flight sim. (Ever played flight sims, and tried to chase down a guy lagging very badly? With his aircraft "blinking" around madly?)
And in addition to that, there would have to be a way to "package" that, and keep tabs on the "when" for each piece of data (without it, you have to assume a constant number of entries for objects... see "additional restrictions above"), and add the contacts etc, which will lead to additional overhead. Look to the mpeg, avi, ogg, and matroska (mkv, mka) multiplexer formats to see what I mean. (Though I guess one could package those features into the entries themselves, it would likely become far less efficient in all ways.)
The most "backward compatible" would be to copy the video codec trick of "key frames". Every five minutes or so, write down the whole state of everything that is in there right then. Then the reader will only have to move on from there.