The expression is English in origin and dates from the 18th century. The first example of it that I can find in print is in the London newspaper The Public Advertiser, December 1767:
How difficult it is even for the Long Arms of the Law to prevent mutual Oppression, Avarice, Ambition and Excess.
In the USA the alternative form ‘the strong arm of the law’ was more widely used, although the ‘long’ variant is found in print there from the 1840s onward. The first use of it there that I have found is from the Wisconsin newspaper the Milwaukie [sic] Commercial Herald, July 1844.
A Mr Neville, of western New York, has married a Miss Amanda Drop, while having another wife. The long arm of the law dropped down on him, and walked him off to prison for bigamy.
Charles Dickens, while not coining the term, did much to popularise its use, in both the ‘long’ and ‘strong’ variants, in The Old Curiosity Shop, 1841:
The gamblers… pursued their course with varying success, until the failure of a spirited enterprise in the way of their profession, dispersed them in various directions, and caused their career to receive a sudden check from the long and strong arm of the law.
See also: the List of Proverbs.