When should I use “ instead of `~~` to join two strings?

Both <>, i.e. StringJoin and ~~, i.e. StringExpression, can be used to join two strings. I know that only the latter can be used to describe a string pattern (instead of joining actual strings). Are there other situations when only <> is appropriate? Or can I always use ~~?