Lose one's marbles

I have a theory (plausible?)about the origin of the phrase "to lose one's marbles"
I suggest that it's the opposite of "knowing your marbles" This was a phrase used to imply a knowledgeable person ewho could identify the source of all the pieces of Italian marble which were inlaid into expensive tables of the
18th(?) century. You can see them at museums... e.g. the Gilbert collection in London. Any comments??

Can you provide any evidence for the use of "know one's marbles" in the 18th century? I have not been able to find it anywhere.
Michael Quinion says "lose one's marbles" is an American expression, which originally meant "lose one's temper", as when losing all one's marbles in a game of marbles. (www.worldwidewords.org/ qa/qa-los1.htm) (VSD)