For the love of Pete!

Does anyone know why we say for the love of Pete and/or for Pete's sake?

It probably refers to Saint Peter. Type "pete" in the Search the Archives box for past discussions.