Skin off my nose

Trying to find where the phrase

'It's no skin off my nose', comes from.

any ideas?

Quotation from Eric Partridge, Dictionary of Catch Phrases: American and British, from the Sixteenth Century to the Present Day: "since c. 1925, at the latest . . . It almost certainly comes from boxing or, anyway, fisticuffs. [Someone else whom the author cites here] relates it to the toast, 'Here's (to) the skin off your nose!'"