[EDIT]
1) It is not directly possible to simultaneously power the usbtinyisp from the USB host and isolate the USB power from the target board.
2) The usbtinyisp is using 5V signaling rather than using a LDO as intended which is a direct violation of the USB standard, and could potentially damage the USB host.
3) The 1.5k pull-up resistors are far too small in the case where we want to do poor-man's level-shifting.
4) Digikey part 151-1081-ND is constructed in such a way that makes it very easy for the connector to short internally. It is flimsy, and is not forgiving with respect to mechanical stresses.
5) The IC2D symbol on your schematic is flipped the wrong way.
6) You are not taking proper care to eliminate digital switching noise on the 2313, and have not decoupled the MCU from the power rails appropriately, creating a condition where a high-current loop across the entire Vcc rail is possible, rather than just between the capacitor and the MCU.
7) Using a resistor alone as a pullup for the reset pin is non-optimal.
Being a fairly high-speed USB device, a resonator could cause more trouble than it's worth when communicating with the USB host; a crystal would be more appropriate.[EDIT]
