I think I really botched the soldering of the crystal, there's a blob of solder on top of the board next to each lead.
With the thin leads on those crystals, that is easy to do. The trick is to heat the solder pad first. When that is hot enough to melt the solder, then let it flow onto the crystal leads.
Contact support with a link to this thread and ask for a replacement crystal. In the mean-time, remove the old one and clean up the excess solder. Some solder-wick might be helpful there.
http://www.adafruit.com/products/149