ACTUV Scoring Structure

[Gargamel]

From the Manual:

You LOSE a point for every 10 seconds you spend at full throttle. (This interval is scaled for progressively lower speeds. As speed decreases, the time between point deductions increases.)From Neal's FAQ:

- fuel pts are decremented 1 point for every 10 seconds at max speed; the time to lose a point increases as the inverse of the square of the speed / max speed
I took this info and charted it out. I wanted to see which sub's were penalized the most for using higher speeds.

The following chart shows Pt's penalized per ten seconds of fuel usage vs ship speed. The lower the number, the less the penalty:

http://img8.imageshack.us/img8/1809/speedpengraph.png

If you can't see it, the Remora and Gator have the same line.

Data:

http://img217.imageshack.us/img217/1282/speedpenchart.png


As you can see, certain subs get a far greater fuel penalty at increasingly higher speeds.

Then from the manual:

• In most missions you RECEIVE 200 points for every minute you hold the SSK on MF /EO/IR/or Radar.
• In most missions you RECEIVE 300 points for every minute you hold the SSK on High Frequency Active Sonar (HF).
• If you hold the SSK on multiple sensors, you get the combined points for all sensors that hold the contact.
So then if we look at our Speed Penalty / minute vs type of Sensor, we get these charts:

http://img141.imageshack.us/img141/3451/hfmfspeedpen.png

This shows how many points per minute we get for various speed levels after taking the penalty/10 seconds into consideration.

So if you read the rules, You get the HF and MF bonuses for using both sensors. So (and I know this is making you nuc sim fan boys holding your head muttering obscenities) if you leave your Active sonar on after you have established contact, you will be practically doubling your score.

This does not take into consideration the pings per minute penalty (You get penalized 1 point per ping), but when you are getting a base of 200 pts for MF tracking, I can't imagine it would take more than 6-10 pings per minute to maintain contact. A fair trade in my opinion. I could get into speed vs MF bonus vs pings per minute, but the graphs would end up 3 dimensional and a real pain to describe here, so I decided to skip it and assume the amount of pings required to hold contact would be the same across all platforms, up to their maximum usable speed, so the differences would be negligible.

So looking at the most efficient ACTUV's, Remora and Gator, we discover that the Gator does not have an active MF system at all. So it looks like the Remora has the potential to be the highest scoring ACTUV.

But...

That's not considering the SSK's actions. If it reacts adversely to the active sonar, then you may end up losing more on the Remora since you will be running around at higher speeds, while the Gator usually just sits behind the SSK using its long range HF. If the Remora can keep the SSK in HF range and under it's usable speed cap (15 kts), I think the Remora would be the best ACTUV, based on the scoring system.

So what's all this mean? Well basically it shows that finding and holding contact is far more valuable than fuel conservation or EM stealth. Getting 200 points per minute for maintaining contact at long range is a far greater bonus than the 6 points per minute fuel penalty and the (guesstimating) 12 points per minute continuous sonar usage penalty. Even worse case scenario, you are earning about 182 pts/min for just maintaining contact, where as a quieter, more fuel efficient approach just accrues fuel penalties.

You start the game with 2000 Bonus points: 1000 bonus points for MF High Frequency Active Sonar (MF) and 1000 bonus points for fuel.

To be complete, this means you can actively ping 1000 times, assuming 12 pings a minute (1 every 5 seconds), or actively run your sonar for 83 minutes and 20 seconds before losing that bonus. You can then run at Full speed for 2 hours and 47 minutes before running out of fuel bonus.

[Molon Labe]

Thanks for that!

I'm a bit concerned about the MF active rule... It doesn't really give you an incentive not to use it a lot... I wonder if that's as intended.

[Gargamel]

Thanks for that!
No prob. I love crunching numbers like this.

I'm a bit concerned about the MF active rule... It doesn't really give you an incentive not to use it a lot... I wonder if that's as intended.I got thinking about that too. I wondered, like you, how accurate the scoring system is to the intended real world application of ACTUV. Google is my friend, Excerpts from the GAO website:

https://www.fbo.gov/utils/view?id=2c557d9073c5c867a1f8de5a09560d70

The Anti-Submarine Warfare (ASW) Continuous Trail Unmanned Vessel (ACTUV) program seeks to develop and demonstrate an independently deploying unmanned surface vessel optimized to provide continuous overt trail of threat submarinesNotice the highlighted part. That screams using your active sonar.

Also:

It is not intended as an ASW search capability. ACTUV relies on conventional force structure to provide a target cue, but then prevents those search assets from being tied up in intensive trail operations.
As the missions have been showing, the idea is for conventional assets (SOSUS, SSN, SSBN, ASW, etc) to make contact with the target, and then hand over control to the ACTUV platform. This explains why the active sonar penalty is so small. Since the asset has handed off to ACTUV, there is little threat to the original asset, so letting them know they've been found is of little concern. In the short term at least.

That also probably rules out the anti-narcosub (ANSW? :D ) theory we had before. Once a primary asset has contact with a narcosub, they are very unlikely to hand off to a drone for contact. Most likely they will either prosecute right then, or hand off to another asset to immediately prosecute. The only scenario I can think of where ACTUV would play a role in ANSW, is where a SSN sits off the the South American Coast, hands off to ACTUV, and let's the drone track the target till they get into waters that fall within US jurisdiction.



• Operations from a shore base
• High system reliability for long duration missions in the harsh maritime environment without opportunity for underway human maintenance or repair

......

• Sufficient range for independent theater or global deployment
• Extended loiter endurance to support forward operational prepositioning
• Speed, maneuverability, and endurance advantage over target set

So this also explains why the fuel penalty is so low. These things are supposed to be designed to have global range. I'm really curious as to the power plant now. Endurance over a SSK after deploying from half a world away? Impressive for a non-nuclear platform.

[CCIP]

In that case it does make the Shark's design rather odd, since it doesn't exactly have an MF setup that would facilitate constant tracking. It's almost impossible for it to keep a target locked up with the MF. The others that possess an MF sonar, though, certainly do seem like constant-ping platforms, especially given the limited (and probably realistic) HF capability.

[Gargamel]

especially given the limited (and probably realistic) HF capability.

Yeah, I wondered about the gator's HF range. 3k? I know I don't know a lot about the systems, but I would have to imagine it's something revolutionary, akin to the AEGIS radar system in it's power output. Slapping on even a 90' or 120' forward looking MF active, while even reducing the HF to 2k would make that the ideal platform.

[CCIP]

Hm, by the way, I'm not seeing a multiple sensor bonus for maintaining both HF and MF at the same time while playing with the Remora here. I do get points for establishing MF contact, but continuous pinging only takes away the 1 point per ping. Are you sure that actually counts as continuous contact? My understanding is that sonar pings are treated as "snapshots", which they are, and will only give you points provided you don't have the sub on HF...

[Gargamel]

I've noticed some discrepancies from the written scoring rules too. I could only do the math with what they provided. I'll run sone tests tommorow and see how it compares. I'd recommend others do the same.

Here's what we'd need to confirm:

HF contacts earn 300 points per minute.
MF contacts earn 200 points per minute.
And the fuel penalty fit the curves as they describe.

For fuel, just run at high TC for an hour at each speed setting and compare your results to my theoretical chart. Even without clarification, we should be able to reverse engineer the scoring engine.

But unless the contact bonuses start dropping to around 50 pts Per minute, the conclusion still applies. Sustaining contact is still vastly more desirable than EM stealth or fuel efficiency.

And what's more important than maintaining contact is not getting too close to another ship. That's listed as -1000 pts PLUS another -1000 pts every 5 minutes.

[Gargamel]

Initial test with the Seahorse running at 20 kts for 30 minutes (should produce just shy of a 125 pt penalty) have been showing results pretty damn close to the epxected result. (123, 124, 124 in the three trials I ran, the error could just be user reaction time to scenario starting up, and the inaccuracy of the clock (1 minute resolution)).

So Even if the contact bonus is not as big as described, as noted unless it is less than 50 pts per minute, the relative value of it does not change. So assuming about 12 pings per minute to maintain contact, here is the priority of your mission objectives:

1) Maintaining a safe distance from other surface vessels
2) Establishing and maintaining HF contact
3) Establishing and maintaining IR/EO/Radar Contact with regard to MF Stealth
4) Establishing and maintaining MF contact, without regard to MF stealth
5) Fuel efficiency

Let me explain 3 & 4. The bonuses for MF, IR, EO, or Radar contact do not seem to stack like the HF one does. So, if the SSK surfaces in front of you, you have contact with it with a sensor other than MF, so you can turn off MF. But if it is submerged, MF is the only sensor (other than HF) that can pick it up, so you have to turn it on.

[CCIP]

12 pings per minute? Just curious, are you doing single pings for that? IIRC, setting it to continuous ping only produces a couple of pings per minute....

[Gargamel]

12 pings per minute? Just curious, are you doing single pings for that? IIRC, setting it to continuous ping only produces a couple of pings per minute....

Kinda a mix of both. I figured I'd use the most extreme reasonable penalty I could imagine, just so there wouldn't be any question about 'padding' teh use of the MF. You can't even match the HF bonus since it's built into the math (+200 vs +300, and you can't have -101 pings per minute, so....)

But there may be a point where pings/minute penalty is less than the fuel efficiency penalty, and that would differ for each platform. But looking at the fuel penalty per minute, even if we consider the opposite, 4 pings /minute, it's only in sprint situations where the fuel penalty surpasses the ping penalty.

http://img171.imageshack.us/img171/959/pingvfuel.png

That's fuel penatly per minute. At the extreme (IMO) end of pings per minute (4), Your MF will probably be useless anyways if the fuel penalty is now worse than the ping penalty.

If you can come up with a chart that shows the active ppm for each platform, please do! I would like to think that we could assume active sonar = constant contact bonus.

[Molon Labe]

Hm, by the way, I'm not seeing a multiple sensor bonus for maintaining both HF and MF at the same time while playing with the Remora here.

The bonuses for MF, IR, EO, or Radar contact do not seem to stack like the HF one does.
:06:

According to the manual, every sensor should stack...
I'm kind of hoping none do. I'd hate to have to ping every minute to max my score for a contact I'm holding with no problems in HF.

[Gargamel]

:06:

According to the manual, every sensor should stack...

Well if this is true, we should try to force him to the surface ASAP. And then keep him there for that would be a 1200 pt per minute bonus. And at that, damn the neutral ships, that's only -1000pts every 5 min.


I'm kind of hoping none do. I'd hate to have to ping every minute to max my score for a contact I'm holding with no problems in HF.

I don't see keeping active sonar on as a major problem.

[Molon Labe]

Decided to contribute instead of comment.

This is on Cat and Mouse and the scoring structure seems slightly different.

Time Score Sensor
1400 unk. PING!!!
1402 2395 HF
1403 2595 HF
.......2695 HF
1404 2895 HF
........2995 HF
1405 3195 HF
........3295 HF
1406 3495 HF
1407 3695 HF
1408 3895 HF
1409 4095 HF
........4195 PING!!!
1410 4394 HF
........4494

So it's giving 200 per minute of HF in this mission and 100 per minute of MF contact, but you are in contact on MF for 5 minutes after the ping.

[Gargamel]

Decided to contribute instead of comment.

This is on Cat and Mouse and the scoring structure seems slightly different.

Time Score Sensor
1400 unk. PING!!!
1402 2395 HF
1403 2595 HF
.......2695 HF
1404 2895 HF
........2995 HF
1405 3195 HF
........3295 HF
1406 3495 HF
1407 3695 HF
1408 3895 HF
1409 4095 HF
........4195 PING!!!
1410 4394 HF
........4494

So it's giving 200 per minute of HF in this mission and 100 per minute of MF contact, but you are in contact on MF for 5 minutes after the ping.

So.....

If we get a 100 pts /minute for 5 minutes per ping, then fuel effeciency becomes more valuable than..... hmm...

I originally thought of this in a linear fashion.... hmmm.... But I think i figured this out.

Here's an algorithm I made for this. To be honest though, most of us just do this intuitively.

http://img854.imageshack.us/img854/6007/darpaactuvflow.jpg


Some of you will argue that HF should be before MF. But since there is not a great reason why you can't establish MF contact once every 5 minutes, then return to HF tracking, MF should be a higher priority than HF. MF is a once every 5 minute thing (at least according to ML's number so far, and my numbers are starting to match up), while HF is constant.

[Molon Labe]

I haven't tested for it, but I'd have to imagine HF contact is also per 5 minutes. Or any other sensor for that matter. As long as the light is still flashing, you're getting the points (or if the light would be flashing, but is steady because you hold contact on another sensor).

[Gargamel]

I haven't tested for it, but I'd have to imagine HF contact is also per 5 minutes. Or any other sensor for that matter. As long as the light is still flashing, you're getting the points (or if the light would be flashing, but is steady because you hold contact on another sensor).

That's probably a good hypothesis too. The bonus for HF is that it's "passive" (compared to the MF). So there's no penalty for using it.

[Gargamel]

Doing some work on a sprint and drift strategy, I came up with this interesting chart.

http://img801.imageshack.us/img801/2825/speedvsscorecolor.jpg

Assuming you have maximum sensor contact on a target (Sensor angle was not considered), it gives you how many points, based on our recent 5 minute findings (and not the written scoring rules), you will earn in the next five minutes given your current speed.

One major note, We have shown that MF scoring continues into the red zone, but that gets really complicated to model in this simple chart, so it assumes that when your speeding, you have lost contact.

Green Indicates maximum scoring with both (or only) sensor, yellow is one sensor, and Red is no sensors.


Couple quick observations from this chart:

1) Once a Gator or Triton locks onto the SSK, it should be practically impossible to shake it off. The maximum sensor speed is greater than the max SSK speed.

2) The Remora is a true sprint and drift boat for intial acquisition, but once locked, it will be difficult to shake. Even if the SSK goes Flank, the Remora can chug along at 7-8 knots tracking it until it too can sprint to keep up.

3) For the Remora and Seahorse, it is imperative to regain HF ASAP, as they cannot be competitive for long with only MF.

4) The triton should be the hands down winner here, from this chart at least, as it can score max points at any speed the SSK can go. But looking at the sensor profile, it requires a deft hand to keep it locked down the whole time.


EDIT: PIC Fixed, had wrong Triton Sensor speeds input. math was right, I was just using bad data. chart is now fixed.

[Gargamel]

So, taking the above data, I compared Speed vs Range and it's effects on the score per 5 minutes for the Seahorse Platform. I will also do Triton and Remora, but I won't do Gator and Shark. Gator is either in the zone or not, and Shark's ranges depend on which way your pointing so it won't be accurate.

And while some may find this chart very intuitive, and ask why we even need it, I did it just to do it. Some may find it useful as It helps them visual the scoring 'bubble' each platform can operate in.

And to be honest, it was kinda fun, trying to work out the following formula was interesting. This had to go in each cell you see:

=IF(CP$2<=$B$33; IF($A14 <= $B$32;1500-(30/(($B$29/$A14)^2)); IF($A14 <=$B$30 ; 500-(30/(($B$29/$A14)^2)); -30/(($B$29/$A14)^2) ));IF(CP$2 <=$B$31;IF($A14 <= $B$30 ; 500-(30/(($B$29/$A14)^2)); -30/(($B$29/$A14)^2) );-30/(($B$29/$A14)^2)))
So for seahorse we get:

http://img233.imageshack.us/img233/6305/seahorsesvr.jpg

Each box is 50 yds, upto 7500 yds out. Sorry bout the hash marks, but when you shrink a number to less than it can show, you get a #, hence why it looks darkened.



Please comment on these, if anybody wants, I can email them the spreadsheet I've been playing with so they can mess around with my numbers.

[Molon Labe]

Gargamel, I think you're right about MF, IR, EO, and Radar not stacking. See this from the debrief file:

Goals
Critical
ACTUV contact on SSK for 30min,Complete,0
Non-Critical
HF contact - 1 minute,Complete,14700
MF/RDR/EO contact - 1 minute,Complete,12000
MF Active Bonus,Complete,986
Fuel Bonus,Complete,918

What we see here is that the scoring triggers are Event Triggers that fire every minute they return True, and that MF, Radar, and EO are all part of the same trigger.

[Gargamel]

Gargamel, I think you're right about MF, IR, EO, and Radar not stacking. See this from the debrief file:



What we see here is that the scoring triggers are Event Triggers that fire every minute they return True, and that MF, Radar, and EO are all part of the same trigger.

Hmm.. then yeah... forcing them to the surface is actually a disadvantage to you, as you cn no longer get the stacked HF + MF score, but just the MF/IR/EO/RDR score.

Not sure how forcing them to the surface is an option anyways :haha:

[Molon Labe]

Well, I think I've pretty much done as well as I'm going to do on all of these (except Gator, I've only played it once and I'm not going back). And it's left me a bit disappointed with the scoring structure.

As was pointed out earlier in this thread, the points given for maintaining contact on both HF and MF are so high in comparison to the fuel and pinging penalties that the decision-making "flowchart" is very straightforward. If you don't have contact on both sensors, getting contact is always your highest priority (other than avoiding proximity to surface traffic), so you never need to concern yourself with speed settings or MF sonar use until you have continuous contact on both sensors. From a gaming point of view, the lack of tradeoffs make it rather uninteresting. From a tactical research point of view, it makes me wonder what DARPA needs us for.... why use crowdsourcing to discover "optimal" tactics when the optimal approach is so obvious?

[dd149]

I can only fully share your opinion, this seems to be a big joke, the tactics are extremely basic and do no seem to offer any challenge in tactics creativity, One can only wonder why DARPA thinks that it could benefit from crowd resources on that aspect. Or is it a creative way to spy on the Subsim community for some reason?:har: In any case, any simmer with some experience of LWAMI or RA modified DW game will not spend much time on ACTUV.

[Molon Labe]

Well, I've spent quite a few hours with it already, but I am at the point where I feel like DARPA needs to give us some sort of information or feedback (or SCS needs to change the missions or scoring structure) in order for there to be a point to go any further.

[TerminatorSub]

Well, I think I've pretty much done as well as I'm going to do on all of these (except Gator, I've only played it once and I'm not going back). And it's left me a bit disappointed with the scoring structure.

As was pointed out earlier in this thread, the points given for maintaining contact on both HF and MF are so high in comparison to the fuel and pinging penalties that the decision-making "flowchart" is very straightforward. If you don't have contact on both sensors, getting contact is always your highest priority (other than avoiding proximity to surface traffic), so you never need to concern yourself with speed settings or MF sonar use until you have continuous contact on both sensors. From a gaming point of view, the lack of tradeoffs make it rather uninteresting. From a tactical research point of view, it makes me wonder what DARPA needs us for.... why use crowdsourcing to discover "optimal" tactics when the optimal approach is so obvious?


You are an expert. :up: I think they also look for newbies like me to add noise to the results.

I Googled some info on optimal submarine search patterns and chose the following papers::know:

faculty.nps.edu/awashburn/docs/MORV6N4.pdf
c3uv.berkeley.edu/papers/Mcgee_acc06.pdf
http://www.wired.com/wiredscience/tag/levy-flight/

It seems the choices to sweep an area are spiral in, zig-zag, or random. I tried these different approaches.

In trailing the target, one can be aggressive (perhaps too much) or lay back a bit. In some of the platforms and scenarios, it is hard to get close enough to acquire a HF lock. That blind spot in the Triton that you mentioned in another post, and the speed limited HF in some cases are factors.

Theory is only good for 5 minutes. Different players will react to the sub throwing decoys and the surface vessel attack in unique ways.

I am wondering if the score results will be analyzed with statistics, or if they will look at each result for fastest target acquisition, closest approach of the surface attack vessel, and shortest time out of contact with the SSK while avoiding said surface attacker?:06:

I also wonder if there will be a second version with more scenarios. The SSK hasn't thrown a torpedo yet, but one should be expected.

[Molon Labe]

I Googled some info on optimal submarine search patterns and chose the following papers::know:

It seems the choices to sweep an area are spiral in, zig-zag, or random. I tried these different approaches.
That's really outside the scope of the ACTUV. The name of the game here is continuous trail. If you need to resort to area searches, you've already "lost." (Mission 1 notwithstanding)


In trailing the target, one can be aggressive (perhaps too much) or lay back a bit. In some of the platforms and scenarios, it is hard to get close enough to acquire a HF lock. That blind spot in the Triton that you mentioned in another post, and the speed limited HF in some cases are factors.
In the real life operations of the ACTUV, this may very well be true. And that's part of the frustration I'm expressing. I really can't fathom why SCS is effectively telling us that it's better to burn tons of gas sprinting around to maintain 300yd range on a contact that's constantly making radical course changes at 20 knots, backing off to reestablish MF contact once every 5 minutes, and sprinting like a madman to get HF back, when you could sit back at 1000yd maintaining MF contact running the diesels at 20 knots or less... probably much less if the target is maneuvering.

But the way this sim is set up, maintaining both MF and HF contact at all times is the tactically correct move, regardless of fuel consumption. And if you absolutely have to lose contact on one (which is how I feel about Triton mission 3, if you're interested), be aggressive and stay with the HF, because you're penalized 50% more for losing HF contact compared to MF.

[StevenLohr]

Molon Labe-First of all, congrtulations. Your scores are awesome. IRT your frustration: While I can't speak for DARPA, I suspect that they are trying to see what solutions that the "experts" (i.e., the subsim crowd) come up with IRT this problem. Over the past 60 years, Operations Analysis (OA) has established the "textbook" solution to the particular trailing problems we are simulating. However, we are being asked to act as a "Red Team" or an alternative analysis group. By evaluating our tactics, DARPA can verify the OA solution.
Additionally, the scenarios we are being given, while simple, are dynamic. The presence of merchant traffic, false contacts and Kamikaze merchants adds a dynamic that OA would have trouble quantifying, but the trained human brain can deal with fairly easily. How we deal with these problems in a dynamic environment gives the programers some insight into how to teach the ACTUV to act. Remember the original "Star Trek" episode where the computer scientist integrated some of his brain onto the computer? As a group, we are kind of doing the same the same thing, helping to train the ACTUV computer to act/react in a dynamic environment.
As an aside, I'm pleased that DARPA is thinking "outside the box" in this endeavor. It is an excellent idea to crowd-sourcing this portion. I would hope that they continue the experiment.

[Gargamel]

I think Steve has it right. IT's not about the tracking methods, as those are pretty cut and dry. Establish contact, don't lose it and get as good a contact as possible.

I think they are trying to see how the drone should react to other factors that aren't directly scored ingame. IE, avoiding merchant traffic. The higher scoring results will have successfully avoided traffic to not get the penalty, and they are trying to see how the better drivers avoid the contact while maintaining optimal contact. If the drone sees the target going for a back scratch manuever on a merchy, how should it react to stay on target and not collide?

What I still don't get though, is the array of sensors and platforms we're being given to work with. Are they seriously using our results to help decide which design bid to accept? Some of the designs are useless at the targets top speed, so throw that out. And then why can one design have uber-sensors, while others have them seemingly pointed in the wrong direction? Why can't they mix and match? If it's a power/size issue, then make the damn thing a bit bigger. I bet some solar panels, wind, and wave generators would work nicely on this thing, etc etc. If they have intentionally designed flaws into the drones to test our tactics, then why the redundancy?

Anyways, With my internet being as it is right now, I haven't been able to submit my results (different machine). I'm just glad I was able to contribute some statistical analysis to the project.

[Molon Labe]

What I still don't get though, is the array of sensors and platforms we're being given to work with. Are they seriously using our results to help decide which design bid to accept? Some of the designs are useless at the targets top speed, so throw that out. And then why can one design have uber-sensors, while others have them seemingly pointed in the wrong direction? Why can't they mix and match? If it's a power/size issue, then make the damn thing a bit bigger. I bet some solar panels, wind, and wave generators would work nicely on this thing, etc etc. If they have intentionally designed flaws into the drones to test our tactics, then why the redundancy?


I've been wondering about that too. If Gator is fictitious, then I'd say the Remora and Shark designs are both very good. I like Shark a little bit more because of the higher MF speed and longer HF range. It's sonar is OP though, at least in the DW acoustic model anything beyond 4nmi is a waste. Seahorse is nice, but I'd rather have the speed of the others than omnidirectional MF sonar. Seahorse's best feature might be the elevation on the MF--although with its long-range HF sonar it's really necessary. Triton blows. It would rock, though, if it traded MF elevation with the Seahorse's. Short HF range should be paired with high elevation MF.