Security implications of download attribute

The download attribute in an a element tells the browser to force the download of a file that otherwise would be interpreted by the browser. This is very convenient, since often users want to download a (e.g. jpg) file instead of having the browser visualise it.

<a href="link.jpg" download="myfile.jpg">Click here to download</a> 

Some browsers block the download attribute when the file is not accessed by the same protocol, on the same host and over the same port. This to me sounds a bit pointless while it breaks a lot of good use cases to prevent something that can be circumvent in other ways.

What are the security implications that browsers try to protect? Any useful real example?