I could care less...

Why do people say "I could care less", as in when Stasko tells me about Republicans, and I say "I could care less" all rolling my eyes and dreading every second of the conversation. Wouldn't the correct thing to say be "I couldn't care less", meaning I am at the very basement of caring, that there is NO WAY I could have less appreciation for what that person is saying. To "could care less" seems to imply that your somewhere in the middle of caring and that your all wishy washy about your concern. Overall, I rate this semisarcistic saying a 3/10 due to its snoodiness and resent controversy.