Yes, I know exactly what you mean, and Subnautica creates that effect superbly.
Around the Aurora, that Abyss, and these huge creatures there with their hair-raising calls.
Going deep into the caves in other habitats, I mean really deep, also has its moments of horror, sometimes. It simply is a really great game experience. The excellence you see in that all this excitement gets created without blood and gore. No massacres. No weapon frenzies. No explicit violence.
Its also one of those games I cannot wait to try the VR mod or patch on. Imagine those kelp forests - in real 3D around you. I imagine that to be nothing but stunning.
Best played all alone, at night, in a dark room. The, sometimes, the abyss really starts to stare back into you. The comic-like screenshots and usual colourful friendly opictures are misleading, they tell you nothign about the real exciting, horrific atmopshere the game sometimes creates. When the light is fading. When the ground below falls deepr and deeper. When you hear "it" calling, and don'T know form where. When you try to track the cave mazes. Even at b egiunning, when you just can swim as free diver and have no real oxygene reserve and wonder whether you make it back to the surface.
Simply a good game. And a very beautifully looking one. And yes, I admit: I so,metimes feel scared, too. Especially at night. Alone. In a dark room.