How to stop unknown spam keyword?

I have a weird problem. Kindly let me know should I be worried or not. I installed a redirect plugin in WordPress and saw a 404 error log. Since last night one Korean bet website was in the list like Korean bet website that resulted in 404 error page. Then I searched it in the string and found it on wp-minify (wp optimizer option) that last night I activated then deleted the wp-minify folder on my WebHost and also deactivate and deleted WPoptimise. After that, I can still see 4,5 links/per minute on 404 error that usually are for spam links (like bet sites, etc..) But nothing on my website. It’s like non existed links to my website with their keywords name. even tho after removing the plugin there is nothing on my WebHost but this link spamming will affect my google score? is there a way I can stop this?

Appreciate any help