Hmmm. 4.2v is the correct output voltage for the charger. And 3.68v indicates that your battery has at least a partial charge.
So I substitute the solar panel with a Benchtop power supply, using 5V.
Leds lights on, the benchtop amperometer remains near 0.
How near to 0? And how long did you leave it connected? The charge controller does not start out at the maximum charge current. It goes through a preconditioning stage before ramping up for the fast charge.
If I plug a load, using an AVR board, the benchtop amperometer signs 20mA.
That sounds like the 'load-sharing' circuit diverting input power directly to the load.