Magento 2 override /lib html file

I am trying to override an html file in the lib folder, but I cant get it to work. Here is what I have so far:

Path to file I want to override: /lib/web/mage/gallery/gallery.html

Path to file I want to override with: /app/design/frontend/{Module}/{Theme}/web/mage/gallery/gallery.html

I have flushed the cache, but it still only loads the original file.

Any help with this would be appreciated.

Update: If I delete the lib files, it will use the files I want to use, but I want to override the lib files, not have to delete them.