Use linux firewall rules for windows too

I’d like to improve security on my home network. I have a PC and a laptop with Microsoft Windows as well as a PC and a laptop with Linux There are a few HTTP services installed in the network. The devices should only be allowed access to DNS to router, HTTP and HTTPS to internet, and HTTP to certain local network devices.

I want to define the firewall rules just once and then roll them out programmatically. Do you know of any good to that allows to configure firewall rules on Windows AND Linux?