I thought I posted this a while ago but I guess I didn't:
10 Ways to Destroy an Arduino
http://ruggedcircuits.com/html/ancp01.html
It's about all the different ways a circuit can be destroyed (in this case an Arduino), and how protection measures can be added.
In terms of making a more robust/safe power supply, I've found that the Murata OKI-78SR is a really, really great little +5V regulator:
http://www.murata-ps.com/data/power/oki-78sr.pdf
It doesn't get hot, operates with 90% efficiency, it supports a wide range of input voltages (+36V), and most importantly it limits current output during a short-circuit condition to only 10mA. A regular 7805 limits the current to 250mA which is still more than enough to fry whatever is connected to it. We've all plugged in EPROMs backwards once or twice (or thrice..) and know how they can be damaged, at least this way they'll come out unscathed.
They aren't cheap, about $4 USD whereas a 7805 is $0.75 USD. But since I've started using these I'd never go back!
Don't forget your two NOPs after CSH.