Lady of the evening

I know what this is, but how did it come to be called this?

My guess is that her visitors would stop by in the late evening, under the cover of darkness.