Before we can get to grips with ‘a different kettle of fish’ we need to know what ‘a kettle of fish’ is when it isn’t different.
The expression ‘a kettle of fish’ means a mess or a muddle.
I should mention at this point that there are conflicting opinions to what the ‘kettle’ being referred to in that phrase was. That needn’t concern us here as ‘a different kettle of fish’ derives from ‘a kettle of fish’ whatever that phrase’s origin.
However, if you want to know more about ‘the source of the expression a kettle of fish‘, click the link.
‘A different kettle of fish’, and I have to say it, is a different kettle of fish from ‘a pretty kettle of fish’. The ‘different’ variant is much later than the original ‘pretty’ form, from which it clearly derives, despite the two phrases have very different meanings.
The expression dates from the late 19th century and was found most commonly in Scotland and the north of England (where fish kettles were and still are quite commonplace).
This early citation comes from a report in the English newspaper the Morning Advertiser, June 1847:
In plain English, a master is responsible for the acts of his servants, and he must either be sent to the right-about by the railway authorities hereafter, or boil a very different kettle of fish.
See also, the meaning and origin of ‘a kettle of fish‘.