Ok, so could you explain why you use 0xFC for the low fuse byte if the external oscillator is supposed to be a 16 MHz crystal? The fuse calculator you linked also claims that the settings you give are for an external crystal oscillator of 3-8 MHz. Your schematic for the atmega32u4 breakout board and the product page both claim that the crystal is running at 16 MHz.
Also, those settings are different than what your atmega32u4 breakout makefile provides here: https://github.com/adafruit/Atmega32u4- ... r/makefile
Only the extended byte is different. But the makefile version of the extended byte setting makes more sense to me than what you wrote here since the high nibble is left unprogrammed (1) and this agrees with what the datasheet calls for.