Key to the manor's back door

 is a quest item and key in the.

After dealing with The Caretaker, this is automatically taken off his body to enter the Von Everec Estate.

Associated quest
Scenes From a Marriage