Storm Kings Thunder Maelstrom Mechanics

Currently running Storm King’s Thunder, and my players just got the conch of teleportation and next session will be using it, the module says you take pressure damage for being at the bottom of the sea.

I’ve looked it up and it seems like most people only count that damage when the players are in the underwater parts. But the entire castle is still at the bottom of the ocean so I’m not sure why being in the underwater part or not would matter.

How did you other DMs handle this? Just curious.