ground cherries. I remember eating them as a kid, but haven't done so in decades. They are a Physalis, related to tomatoes.
Chime in if you know anything about these. I'd love to know more.
UPDATE 12/5/13: Thanks to reader C Helms, we have another option for these berries. They look an awful lot like Horse Nettle, Solanum carolinense. No husk.
The big difference seems to be that the physalis ground cherries have a husk that looks a lot like a paper lantern. Horse nettles are often thorny - and they have varying levels of toxicity, so be careful before eating any.