Implicitly allow requests in IIS from valid hostname

I have a few publicly accessible IIS servers and sites (personal and corporate), these hosts have own domains/subdomains, and all legit access to these https sites happen through domains.

Almost all HTTP app vulnerability scans from bots/rooted servers happen to the servers through IP, without valid hostname, and if there is hostname it is the default reverse DNS host, not the actual domain of the site.

Is there a way in IIS to implicitly only allow requests with proper hostname? The site root app only has bindings to the hostname, but IIS still accepts requests, and responds with 404. The best thing would be to timeout the request similar fashion as if the site doesn’t have HTTP open.

I of course understand that this does not guarantee anything in security wise, the scanner can still figure out the proper hostname in many ways, but it would still filter out 90% of dummy scans.

IPS in firewall can probably do some things, but in some cases I do not have that luxury. Is there way in IIS? Redirect the http request to oblivion? (this would probably just change the error to proxy gateway http errors?)