Does rolling AC to oppose an Attack Roll impact game balance?

I really like the concept of rolling d20 against each other during ability checks like grappling, etc. I want the same thing for normal attacks, so the defender would also roll a d20 + (AC-10). Would this modification make the game somehow unbalanced or create side effects on monster challenge levels?