Haste and Casting a New Concentration Spell

A spellcaster currently concentrating on haste on themselves sees a situation unfolding that desperately requires them to concentrate on another spell, for example, hypnotic pattern.

They move and take their action to start casting the new spell. However, concentration on a spell begins once you start casting it, which would immediately end haste, which causes lethargy making the spellcaster unable to move or take actions until the start of their next turn.

The spellcaster has already finished moving, and has already chosen an action to take (casting a new concentration spell), and has actually started that action.

Does haste’s lethargy effect interrupt the casting of the new spell, or does the spell go off and then immediately after the caster is lethargic?