Surprising an alert character

My character with alert feat was attacked and received damage pre-combat. Specifically, our party was walking through dark cave, and on the ceiling there was a swarm of insects (that none of the PCs noticed). Suddenly, DM describes that a swarm of insects lunges at my character and immediately attacks him dealing x damage. I asked him why the moment the swarm tried to attack me he didn’t ask us to roll initiative to see who goes first. He said that was because the swarm noticed us and readied its action to attack me when I come in range. Can an alert character be surprised in such a way? Can an enemy who wants to get the drop on a party bypass rolling for initiative versus alert character by reading attack action?