I intend to update this test mod in the somewhat near future.
A few questions for anyone who might know:
1. will the game recognize 2 of the same general sensor type installed? I tried cloning the O node on a DD, and renaming it O02 to add a 2d visual sensor, but it didn't seem to work. It could very well be a failure in my ability to properly work the nodes, however. The goal here would be 2 concentric AI_visuals on 1 ship. One longer ranged version, and one very short ranged version. In my tests I didn't have them overlap, so one goes from 2000m to max rannge, the other goes from 0 to 1999m.
2. Any insights into proper settings for AI visuals to detect a certain cross sectional area at a given range X% of the time?
3. Does "alert" status for AI ships alter their detection probabilities at all?
4. The default game has the "use crew skill" (whatever it's called in sensors.cfg) set to false. What's the effect with this on?
Musings about the requirements for such a mod in terms of realism:
Strictly speaking, with no consideration but a detector at the highest point on the ship, detections ranges should be 100% reciprocal. I see your eyeball, you see mine. That said, the eyeball in space at the tip of the mast is not reality. This boundary case does apply to first detection, however.
The detector (the watch crew) are not at the very top of the ship. So all things being equal, the ship with the largest cross sectional area above the watch crews gets detected first. I am in the shears with 2m^2 of metal above my eyeballs, and the Fuso I'm observing has 20m^2 of pagoda above the observer there. I suppose we could get into typical resolution limits for the naked eye, or even normal telescopes (those without adaptive optics) and figure out what minimal cross section should be invisible at a given range—actually, that might be a useful calculation for a boundary condition... (it'll give me a chance to brush up, I'd almost forgotten I was an astronomer in training once

)
Luckily, for submarines this is somewhat straightforward, anyway. The guys in the shears are still close to the water, and the cross sectional areas involved are tiny in the case of a sub compared to a ship. The short answer is that a submarine should, barring a human failure, ALWAYS visually detect a ship first. This is particularly true if you consider smoke, which functions as a large, detectable cross section (even for pretty non-smokey ships) that resides well above the ship's watch crew.
Seriously, I cannot imagine a realistic case where this is not true, with the only exception being crew skill. Put a myopic guy without glasses in the shears, and all bets are off (assuming tojo in the BB's mast isn't the coke bottle wearing ww2 stereotype), but assuming equal AI skills, the ship should always be seen first IMO. The game limitation of visual limit IS an issue here, because even set to 16km, the sub would have detected the ships outside the visual limit (capital ships, anyway). At 7-8km, we are in a situation where the targets may very well have already detected the sub in RL, but the sub should still be aware of them first.
Once inside the extreme range case (top of masts or smoke on the horizon), things change a little. Once both ships' watch crews are well above the horizon to each other, you have to take the cross sectional area above the horizon (to the other observer) into account. So the next obvious test case would be a range at which the sub is entirely hull up.
It's still a small target, so the chances of any given set of eyeballs on the ship looking for it should be lower than the chances for the sub to see the part of the ship above the horizon. The ship "wins" in terms of simple geometry (higher mast), but the cross section of the sub is smaller than the hull-up portion of the ship. The difference happens when you look at watch crew numbers, and their eqipment. At a certain point, the ships start having more watch crew. So the smaller instantaneous chances of detecting the sub are mitigated by more eyeballs. Better optical gear could have an effect as well.
In general, I'd expect the sub to always spot ships visually first at extreme range (clear days). There are
zero plausible exceptions to this, IMO.
Once the sub is hull up to the ships, it becomes more complex. The ships should always be visible to the sub, obviously, but their chance of seeing him should increase. As more of the ship is hull up, more eyeballs come to bear. It might be interesting to know how many watch crew a typical IJN DD stood vs a CA, BB, etc. Comparing that to a sub's watch would give a relative number of eyeballs, and that would scale in some fashion to detection.
Right now internally I have 4 AI visuals (or 5, depending on which version I use, lol). I might increase this a bit.
My thought was to set them primarily based upon the expected number of watch crew, which would also scale to ship size/height (all things bein equal, the higher the watch, the better).
IJN BBs had watches up as high as 150 ft, though perhaps 100-120 was where the bulk of them would be. CAs perhaps 60-75'. DDs have what looks like a crow's nest in the mainmast at ~60', but that would be only a few of the watch crew, the bulk look to be more like 30', tops. Subs look to have a watch someplace in a similar range up in the shears, 20-30'. Merchants have some height to work with, but in most cases they'd simply not have the large number of trained eyeballs as a warship.
So I'm thinking (some should be combined if they are similar in effect):
1. Merchant (few watch crew (30 - 45' max watch, few crew))
2. Merchant 2 (more watch crew (45' - 60' max watch))
3. Small warships (subchasers (45' max watch))
4. DD (60' max watch)
5. CL/CA (CVs would be here, too (75' max watch))
6. BB (150' max watch)
7. Single crew aircraft
8. multi crew aircraft
They are arranged in order of detection probability/distance. All but the aircraft should have the sub detecting the ship first, no exceptions. The planes are smaller, and on top of that occupy a large volume of space instead of a linear position on the horizon. Getting jumped by a plane if you don't have radar should be likely, getting jumped by a ship in clear weather should be completely impossible.
2 and 3 should probably be combined. I might use that level only on certain merchants of high value. That would make 7 visuals. A simple way to create variability for warships would be to "BP-clone" warships to make an additional class for a few. The class would get a ship name within the real class. The clone would be identical except for having the visual set 1 notch lower. Then you might come across 2 asashios with std DD visuals, and 1 Kagero, Isokage, whatever, with "small warship" visuals (crew had too much sake on liberty

)
The goal would be accurate
outcomes, not strictly accurate visual limitations since there are simply not enough variables to work with set them to what we know a RL limit is, and they might be superhuman in game in terms of spotting.
To start I will assume the stock view distance, for example. The benchmark is sub detection for the AI, and ship detection for the sub. The sub should always see ships first, and the bigger ships should pick up the sub faster than the smaller ones all things being equal. We know from patrol reports that it should be possible to conduct night surafce attacks, and even that some subs got within under 1000 yards of escorts undetected. This should be possible with good sub handling, but it should not be automatic, IMO.
I'll add a couple more to my test.
Note when I release this as a test, I will likely simply make a set of ships with 1 of each sensor type, and a test mission for you. To control the test, I will eliminate all other sensors from the test ships so we can get decent results.