Unfortunately I find myself in the exact same situation. I did do the initial setup with my USB stick plugged into the top of the back two ports - having never taken the cover off my v4, I thought this is what was meant by the top USB port! However, the entire process seemed to work without a hitch, up to the point of rebooting, of course.
Since then, I've tried booting the same stick in the proper port on the top - no dice. I even tried going through step 9 again in case alarm wasn't installed properly on the USB stick. Still no boot.
The behaviour of my PogoPlug is that it flashes green LED on the front after turning on, and continues to flash, while the LED on the stick itself flashes briefly then stops, the green LED still flashing... it just doesn't stop flashing ever. It never boots properly.
I'm lost 
 EDIT:
EDIT: Successful reversion! 
If you are using the same USB stick to revert, MAKE SURE you reformat it as FAT32. If you leave it as ext3 (as required by the installation instructions) and simply delete everything and make a "revert" directory, it WON'T work. The official instructions to revert do say FAT32 of course - I just missed that bit 
