Storing private keys for updates to remote device

I’m reading up on how to perform signed updates for remote hardware devices. I need to check if the new software has been generated by a "trusted" source, ie me.

Based on my understanding of asymmetric cryptography I understand that I can embed a public key on all my devices and then any new software that needs to be updated on the device, needs to have a signed hash in a header that I could check against. I’m all onboard until this part, and it sounds great for secure updates, but I’m wondering how one would securely store the private key. Say I built 1M units. All have the public key programmed on them. If my private key is lost for whatever reason, I lose the ability to update any of these devices. It sounds like the jugular vein. Should I accept storing private keys very safely as a cost of doing business, or is there a better way to handle this case?