Hats/Hat's off

Which is correct? "Hats off" (without the apostrophe)wins the Google fight, but isn't using "Hat's off" as a shortened version of "My hat is off to you", acceptable?

But you could also make the case for a shortened "everybody take your hats off for ..." Could be plural.