The phrases "on the contrary" and "to the contrary" are used to reply to an opposing point. Your friend tells you she is moving to New York and you express surprise because you thought she hated big cities. She replies, "On the contrary, I've always wanted to live in an urban area."
When a distinction is being made that does not involve opposition of this sort, "in contrast" is appropriate. "In New York, you don't need a car. In Los Angeles, in contrast, you can't really get along without one, though you won't need a snow shovel."
Here's a simple test: if you could possibly substitute "that's wrong" the phrase you want is "on the contrary" or "to the contrary" If not, then use "in contrast."