When looking through the combat rules I realized under “Being Prone” it states:
You can drop prone without using any of your speed…
There doesn’t seem to be a reference that this needs to be on your turn.
(Note, even if you wanted dropping prone to require movement because it is under the “Movement and Positioning” section, there are ways to move off of your turn such as dissonant whispers or the Ready action)
Can you drop prone when it is not your turn?