Throw in the towel
This little expression of course derives from boxing. When a boxer is suffering a beating and his corner want to stop the fight they literally throw in the towel to indicate their conceding of the fight. This earliest citation that I have found of this is in the American newspaper The F ort Wayne Journal-Gazette, January 1913:
Murphy went after him, landing right and left undefended face. The crowd importuned referee Griffin to stop the fight and a towel was thrown from Burns' corner as a token of defeat.
It was very soon after that that the phrase began to be used in a figurative sense, to indicate giving up in non-boxing contexts; for example, in the Australian author Clarence James Dennis's WWI patriotic novel, The Moods of Ginger Mick, 1916:
No matter wot 'e done. It's jist a thing
I knoo 'e'd do if once 'e got the show.
An' it would never please 'im fer to sling
Tall tork at 'im jist cos 'e acted so.
"Don't make a song uv it!" I 'ear 'im growl,
"I've done me limit, an' tossed in the tow'l."
Throwing in the towel was preceded by throwing in the sponge. Sponges were a common ringside accessory as early as the 18th century. Throwing in the sponge was then the preferred method of conceding defeat. This is recorded in the mid-19th century, in The Slang Dictionary, 1860:
'To throw up the sponge,' to submit, give over the struggle, - from the practice of throwing up the sponge used to cleanse the combatants' faces, at a prize~fight, as a signal that the 'mill' is concluded.