What is the risk of having a 2FA key permanently plugged into my device?

Yubico offers the YubiKey Nano, a 2FA key designed to be left inside the device more or less permanently.

enter image description here

While it does add comfort to be able to just leave it plugged in, what risks would there be if the device was stolen?

From what I could gather, local device accounts would have the same level of protection as a regular passphrase would provide. Online accounts, depending on the setup, would either have no protection at all (e.g. through a “Remember me on this device” function), or the same protection as a regular passphrase.

Is there anything I am missing?