Thank you for understanding Targor :salute:

...sometimes :D

I have finished enough of the AI testing to post the results. In the end, I completed 90 tests, which is short of my goal but hopefully sufficient to still have statistical relevance. You will see that I only ran ten tests at the "poor" skill level, since further tests seemed like a waste of time; I will explain why below.

I have attached the data sets to this post. (By the way, how do I insert images directly into my posts?)

Testing Parameters
All tests were performed in the same mission with only the skill level altered between tests. Further, I performed the tests for each skill level in groups of five at a time to ensure I did not accidentally alter any of the parameters of the mission when I changed the skill level. The U-boat was positioned approximately 3 km away from a convoy consisting of one large liner and five medium merchants. I ran a number of tests with various escorts, but I decided to use a best-case scenario as my baseline test. As mentioned yesterday, I was only testing outcomes, rather than following each torpedo fired. I gave the AI U-boat one in-game hour to sink as many ships as possible, assigning one point for sinking a ship and half a point for damaging a ship but not sinking it. Whenever a ship was damaged, I observed it for one additional in-game hour to see if it would sink.

Results and the Competent/Veteran Anomaly
As you can see, "competent" is still scoring slightly better than "veteran." After observing the AI U-boats and the convoys they hunted, I've come to the conclusion that the veterancy levels mainly affect U-boat sensors. For example, with "poor" skill level, the U-boat never noticed the convoy. On several occasions, the boat approached the convoy, passed between the columns of ships, nearly collided with the large liner, and then sailed merrily along, with the commander apparently none the wiser that he had passed within about 10 meters from a 30,000 GRT ship! In the case of "competent" scoring higher than "veteran," I believe this is to do with the fact that the competent commander noticed the convoy later than the veteran commander, which meant that he was closer to the convoy and able to take more accurate shots. Anecdotally, my observations appear to bear this out: the vast majority of the successful attacks by competent commanders occurred during their first attack on the convoy; unlike competent commanders, veterans and elite captains sank most of their targets after the initial torpedo salvo, as they chased, reloaded, and fired again. The competent commanders, while more accurate for the first shot, had difficulty tracking the convoy once it took evasive action.

Although this reveals some interesting insights into the AI, it also points to a shortcoming of this testing method. Since veteran and elite commanders are better at detecting the enemy in the first place, they would likely continue to get high marks even when placed farther away from the convoy. Competent commanders, on the other hand, would see their effectiveness decline rapidly the farther they got from the convoy, since they might not even be able to sense it.

Proposal
In light of these findings, I propose we stick to a distribution of skill levels that mostly ignores the anomaly between veteran and competent skill settings. I also think we can ignore the poor and possibly the novice skills, since those settings make AI U-boats quite bad. In the scoring rubric below, I assigned points to 1) commander ranks and 2) tonnage sunk. Tonnage sunk is given more weight than rank, since practical experience sinking ships would likely make one a better U-boat commander than wearing shiny epaulettes. These point levels, when added together, give us a corresponding veterancy level. I don't have much experience creating Excel formulae, so the point system below might seem a bit crude. I'm sure there is a simpler and more elegant way to do it.

Ranks:
Oberleutnant z. S. = 1 point
Kapitänleutnant = 2 points
Korvettenkapitän = 3 points

Tonnage Sunk:
1 - 15,000 = 2 points
15,001 - 25,000 = 3 points
over 25,000 = 5 points
--------------------------
Competent = 1 point (default)
Veteran = 4 points
Elite = 6 points

I have also tried to create another iteration of the scoring rubric for the later war, since it seems that the quality of training, as well as the opportunities to make mistakes and not get killed for them, would have decreased. I was thinking about something like using the above system from Sept. 1939 until late 1943 and then using a second system, possibly with "novice" as the default skill setting, until the end of the war. What do you guys think?

Nice! Very interesting results!

Very good job! :up:

Well done, Daniel :up:

Based on your findings, this is my proposal for crew ranking calculation:

lesser than 12,500 tons sunk: Poor
12,500 - 24,999: Novice
25,000 - 37,499: Competent
37,500 - 49,999: Veteran
50,000 and more: Elite

Commander's ranks would contribute to the total tonnage sunk with the following additional factors:

Fähnrich zur See: 3,125
Oberfähnrich zur See: 6,250
Leutnant zur See: 9,325
Oberleutnant zur See: 12,500
Kapitänleutnant: 15,625
Korvettenkapitän: 18,750
Fregattenkapitän: 21,875
Kapitän zur See: 25,000

As you can see, the "Poor" ranking would be extremely unlikely, being applied to commanders with not tonnage sunk whose rank was lower than "Oberleutnant zur See" (the most common rank of U-boat commanders, according to uboat.net). "Novice" would be the most common in-gane ranking applied to medium-high rank commanders at the beginning of their careers. Getting promoted to the "Elite" level would take 37,500 tons sunk to an Oberleutnant and 34,375 to a Kapitänleutnant; respectively 25,000 and 21,875 for the "Veteran" level, and 12,500 - 9,375 for the "Competent" level. On the other hand, the maximum rank which a Kapitän zur See could attain having sunk lesser than 12,500 tons, is "Competent" (which is also his starting ranking); but it is extremely unlikely that such an high-ranked officer would have been given the command of an U-boat anyway.

What do you think? :hmm2:

Wouldn't better ASW sensors and AI be enough to realistically decrease the performance of our virtual wolf pack mates, at late war? :06:
Moreover, the assignement of many unexperienced commanders to U-boat's command during late war, would further decrease their performance...

Your system is much more elegant than mine. I like it, except I think it might give us too many novice commanders at the beginning of their careers. Even when placed in an ideal position against a totally unguarded convoy, novice AI is not very good. My concern is that, if forced to intercept a guarded convoy from a greater distance, novice commanders would be almost useless.

Also, I wonder if there should be a bonus to prewar U-boat commanders. Many of these captains had been the beneficiaries of extensive training programs while the U-boot arm was being built up. A number of them also participated in interdiction patrols during the Spanish civil war and conducted simulated wolfpack attacks beginning in late 1936 in the Baltic and in the Atlantic proper. This is why I was inclined to make "competent" the default setting. Then again, I might just have difficulty calling Prien, Lüth, etc. novices... :hmmm:

Good points :yep: Although I did not record any instances of destroyers sinking AI U-boats with depth charges, their very presence was often enough to allow the convoy to escape.

Let's make a few examples, based on my proposed ranking system:

U-27 / Johannes Franz

U-28 / Günter Kuhnke

U-29 / Otto Schuhart

U-30 / Fritz-Julius Lemp

U-31 / Johannes Habekost

U-31 / Wilfried Prellberg

U-32 / Paul Büchel

U-32 / Hans Jenisch

U-33 / Hans-Wilhelm von Dresky

U-34 / Wilhelm Rollmann

U-35 / Werner Lott

U-36 / Wilhelm Fröhlich

Notes:
- when not inferable from uboat.net's information, Oberleutnant zur See was used as standard commander's rank.
- ships sunk by mine or captured weren't considered
- ships damaged and warship tonnages were considered as "normal" tonnage

Comments:
Maybe we should apply progressively wider tonnage ranges for higher rankings :hmmm:

Some very illustrative examples. Thanks gap :up: How about slightly decreasing the requirements for the competent ranking but increasing them for veteran and elite? That way, more commanders will fall into the middle categories, meaning it would be be easy to gain initial experience but it would become progressively more difficult to achieve a higher status.

What about the following figures?

lesser than 6,250 tons sunk: Poor
6,250 - 12,499: Novice
12,500 - 24,999: Competent
25,000 - 49,999: Veteran
50,000 and more: Elite

Commander's rank additional factors:

Fähnrich zur See: 1,562
Oberfähnrich zur See: 3,125
Leutnant zur See: 4,687
Oberleutnant zur See: 6,250
Kapitänleutnant: 7,812
Korvettenkapitän: 9,375
Fregattenkapitän: 10,937
Kapitän zur See: 12,500

They should make unexperienced U-boat crews to gain experience much faster at the beginning of their career. The novice level would still appear, but much lesser frequently :yep:

a few remarks:

1. rank-relative additional tonnages in our ranking system are exactly meant to mimic this fact, giving a starting bonus to the most experienced commanders. If you look at my previous post, at the very beginning of the conflict the most common rank was Kptlt. Later during the war, low-ranking officers were assigned U-boat commands more often, due to the high casualty rates.

2. Tonnages sunk during the first part of the war were consistently higher than at later stages. This should make successful commanders (such as Prien, Lüth, etc.) to achieve high ranking levels in few patrols.

3. During the first months of the conflict, group strategies were hardly used. By the time the wolfpack tactic had been fully developed, our aces should have attained their maximum rankings :up:

Another remark:

During late war, tonnage success decreased drastically. This fact will be refected by our ranking system with lower veterancy levels, without need of further adjustements.

Using the new ranges:

U-27 / Johannes Franz

U-28 / Günter Kuhnke

U-29 / Otto Schuhart

U-30 / Fritz-Julius Lemp

U-31 / Johannes Habekost

U-31 / Wilfried Prellberg

U-32 / Paul Büchel

U-32 / Hans Jenisch

U-33 / Hans-Wilhelm von Dresky

U-34 / Wilhelm Rollmann

U-35 / Werner Lott

U-36 / Wilhelm Fröhlich

Result:

1 Poor + 9 Novice levels versus 16 Novice levels obtained for the same U-boat commanders using the previous ranges :03:

I like your figures and am convinced by your arguments :up:

One final question: do you think we should use the poor skill? So far poor commanders have not even noticed the convoy was there, even though they sailed right beside it.

Personally I would keep them for three reasons:

1 - "Poor" rankings will be quite rare, being applied only to unusually low-ranked U-boat commanders. I cannot think of a Cadet or a Senior Cadet being given the command of a submarine other than in emergency circumstances, whereas a Leutenant would need to sink just 1,653 tons for being promoted to the next level (which is usually achieved in one, maximum two war patrols).

2 - Loving variety, I think that giving up one veterancy level out of five would be a shame.

3 - Possibly, there is some space for making "Poor" subs more effective, via AI or sensors tweaks. Or at least I hope so :hmm2:

Sounds good to me. As long as it is rare, we can use it to add variety. As you said, there's also no reason to think U-boat AI could not be improved, judging from TDW's successes with IRAI.

Okay, tomorrow I will start implementing the new U-boat ranking system. Keep me informed in case you get any other remark :up:

Sounds good. I will let you know if I can do some more testing, but I think I need to devote a few hours to my thesis tonight :shifty:

Never mind. Your thesis have the priority, indeed :)

done. :up:

At this point I got just to make some little changes to the torpedo/fuel fucntions, and to add the custom base coordinates :up: