Do you need the username and password to remotely manage all admins via ARD?

Or do you only need the username and password of one admin? I share a computer and we are both admins, but I am worried that the other admin could access my files via Apple Remote Desktop. Is that the case or would he need my password to access my user account and files via ARD?