How does Blink work versus a readied action in this PvP scenario?

Bit of an odd one but I want rulings for this scenario:

Yes this is PVP and it ending up been fun at the table, and no one was upset.

Playing a Level 7 college of sword Bard with a Homemade Magic item that gives me access to the Blink Spell.

Fighting a Level 7 Paladin who is a new party member and wants to test their skills.

We are sparing with each other, the goal is first to score three hits or to KO the opponent.

During the fight, the Bard casts Blink, ends their turn, the roll is lucky – he gets a 15 and enters the ethereal plane.

The Paladin not knowing what to do ready’s an action to attack when they reappear.

On the Bards turn, the Bard reeappears and attacks. The Bard is a dual-wielder and has multi-attack.

The DM rules that the Bard gets their first attack, then the Paladin gets their ready action to attack back. After that, the Bard can then take their second attack, and then their bonus attack, if they wish.

Note that the Paladin had cast a spell a few turns ago that on their next hit the target has to pass a Wis-save or become frightened of them – I can’t remember the spell or ability

The first attack by the Bard is a miss.

The Paladin hits back and misses as well.

The Paladin uses a second Attack which the DM allows them before the Bard gets theirs and hits, the Bard fails their save-throw, is now frightened and at disadvantage.

The Bards second attack misses at disadvantage.

The Bards off-hand-attack misses at disadvantage.

The Bard rolls their blink, but the roll is low, so they stay.

The Paladin takes two attacks and both hit, accumulating three hits and is declared the winner.

We laugh and move on.

I think it should have gone different. The Bard should have been able to use all three of their attacks before the Paladin got any, and the Paladin should have only been able to attack once – Since a second attack should have only have been on their turn.

It might have made no difference, but the attacks could still have missed. I felt that the bard, who used a smart tactic of Blink – using their speed over the Paladins strength, was punished by the DM.

Did my Dm make a correct ruling?