To use the battery as a failover device, and charge it when not in use, you just need a couple of diode and resister. A voltage regulator is also a good idea.
The only concern I'd have with a small battery is finding one which can provide the required power but if it's possible to find one which could be built in without making the case bigger, it would allow the Bunny to be used in plenty of other attack scenarios.