Man of the cloth

Friends,

I am looking for the origin of the phrase "man of the cloth" used with regard to clergy.

Thanks!

I couldn't find this in the archives or in my notes. But I recall finding the origin of this phrase in a reference book. When I figure out where I read this, I'll post the exact reference.
Anyway, from memory: At one point, any man wearing the "uniform" of his profession (for example, a baker wearing an apron and chef's hat) was called a "man of the cloth." Eventually, this term only came to mean a member of the clergy.