A "sparkle pony" is slang (possible origin: Burning Man) for a festival-goer who is not realistically prepared for the event; consequently, it's an appropriate name for the solo artist played in this episode by The Decemberists' Jenny Conlee.
Corin Tucker, who appears in this episode as a member of "Echo Echo," sings and plays guitar along with Carrie Brownstein in the acclaimed post-punk rock band Sleater-Kinney.
"Sparkle pony" is a play on the musician Mark Linkus from Sparklehorse.