I've got your number

As if to say, "I know what you're up to" or "I've got my eye on you" - where did "I've got your number" originate?

Perhaps from knowing the number of the house where the person lived? Remotely, the telephone number, but I think the phrase is older than the telephone.