Morticia talks about taking a new surname. She suggests Jones (her real surname - Carolyn Jones) but Uncle Fester scoffs and says "what kind of name is that?"
After Fester blows up the safe, the first thing Morticia takes out is the "thing" they bought for $250 from a scavenger in the episode "Thing is Missing" (s01e23).
The painting in front of the safe is a portrait of the 21st President of the United States, Chester A. Arthur.