You need a load: something that sits between the positive and negative wires from the USB cable besides the ammeter.
An ammeter is basically a short circuit (it has very low resistance), so you want the voltage difference across the probes to be very low. Putting just an ammeter between wires whose voltage difference is 5v Would Be Bad. You need something else (the load) to fill up that 5v voltage gap while still letting all the current through.
For 2.1 amps of current at 5v, you'll have to find a load whose effective resistance is about 2.4 ohms. That will take some effort.
The simplest way would be to get 18 1/4-Watt 10-ohm resistors. Make two bundles, each with 9 resistors in parallel. Connect one end of the first bundle to the positive wire from your USB cable, and one end of your second bundle to the negative wire from your USB cable. Then connect the ammeter between the bundles.
Each bundle will be equivalent to a 2-1/4-Watt 1.1-ohm resistor, and the series resistance across both bundles will be about 2.2 ohms. 2.1 amps of current should produce 4.65v across that resistance, which is close enough to the regulator's 5v nominal voltage that it shouldn't cause any problems.
When you void a product warrany, you give up your right to sue the manufacturer if something goes wrong and accept full responsibility for whatever happens next. And then you truly own the product.