If you have cast Spiritual Weapon and then you are polymorphed, can you make melee spell attacks?

I have a player that would like to cast Polymorph on the party’s cleric.

If the cleric has already cast Spiritual Weapon on himself (this is not a concentration spell) and he is Polymorphed into say a Giant Ape, does he still have the ability to make Melee spell attacks as a bonus action?

I know that you can continue to concentrate on spells that you cast before you Polymorph, and that you cannot cast new spells. But would a great ape be able to make a “melee spell attack”?