Driving me Bananas

Nobody laugh but I have been going nuts driving to find the origin of the phrase "Driving me Bananas" Someone brought up the challenger and I have been looking ever since. If anyone could help me with this I would be very greatful. Thank you