AO = Ambient Occlusion. They're the textures (.O0X extension) that apply static shadows/lighting to various object in game, i.e. ships, subs, weapons, etc.
Here's a basic tutorial I wrote up a while back detailing how to create AO maps, should allow you to get started.
Ambient Occlusion Tutorial