Understanding Secure Boot

I’m trying to understand the secure boot process of an OS but there are few points I can’t wrap my head around.

At a high level, afaik, secure boot ensures that the loaded OS is authenticated by its respective vendor. If an adversary modifies the OS code, the authentication checks during secure boot fails and user is notified.

What I want to understand is how’s this mechanism implemented at a low level. My understanding is as follows.

There’s a read-only memory (ROM) where the program which initiates the booting process is written along with a public key by the manufacturer. Integrity of this code is basically implicitly trusted, so this program is basically the root-of-trust. This program is loaded by CPU first and upon execution, it verifies and loads the next component in the booting process. Next component verifies the next next component and so on until all the components of OS are loaded.

However, what ensures that CPU really starts booting the system from the correct ROM? Can’t an adversary force the CPU to read a malicious initiating program that disregards the verification step? That is, there should be another component that ensures the system really starts from the root-of-trust program. What’s that component or is my understanding of the process is incorrect?