Poe folk don't 'spect much?
In an episode of “Frasier”, Frasier is worried about his upcoming recitation of “Annabel Lee” to a local Poe Society. His brother Niles says to him: “Don’t worry. Poe folk don’t ’spect much.” Hilarity ensues. Clearly both Niles and the audience thought he'd made a delicious pun.
I'm not used to being the dull knife in the drawer, but I don't get this one. I thought I had it when I imagined it punning on a phrase like "poor folk don't 'spect much", which sounds like a likely quote from an ancient play, but no luck, it does not seem it has ever been uttered until I thunk it. (It should have, it's a good phrase).
Help me out?