Implicit conversion: when does it hurt performance a lot?

I know that when data types are the same it is good and it hurts performance when they are not.

But sometimes it seems to have very little impact and basically a waste of time to fix it. Other times it will have a huge impact and performance gets a lot better.

Of course there are many scenarios, e.g. a insert select where the data type of the select is not the same, but compatible, with the insert column. Or it could be on the join predicates.

Where does implicit conversion hurt the most?

Also it seems that one also gets an implicit conversion when the lengths don’t match, e.g. joining varchar(10) with varchar(20). How big a deal is these scenarios when we are only dealing with the length of the columns and not their type?

Basically I would like to know when I should worry about the compute scalar operator in the execution plan. How can I know if removing it will have a significant impact?