America’s Oldest Record Store? America’s Oldest Record Store? Depending on who you ask, one’s in Pennsylvania, the other’s in New Jersey. Both places matter. The