I'm not the biggest consumer of scare games, but
Dark Fall: The Journal greatly impressed me with everything except its idiotic name. It's exactly as you described in the OP: no FPS stuff (there aren't any enemies and you can't, in fact, even die) and exploring a big hotel with a gripping backstory.
It's more about the whole atmosphere than quick scares, so there are no "boo a ghost behind the corner!" scenes either. The puzzles are usually logical and rely more on traditional problem solving (such as figuring out a secret message's code) than trying every item you find on every possible object in the world, hoping to guess what the developer wanted you to do.
I didn't get really far in the sequel, but it didn't seem bad either. I just wasn't in the mood after just finishing the first game.
Edit: remembered wrong, it was a hotel and not a mansion. Should start reading my own Wiki links
.