As a hardcore Morrowind/Oblivion vet, I kicked the difficulty up to the max from day one. I've never had a problem with the game. Indeed, I found it too easy. I'm not saying this to brag or boast. I'm really just whining about games being too easy these days.
Or maybe I've just achieved CHIM and realized that any and all damage to my character is just a figment of the Godhead's imagination.