When to implement authentication and authorization?

I am currently in situation where I have to decide when to implement security measures. So the question is simple, is it better to implement authorization and authentication methods right off the bat in the beginning of the project, OR at the end of the project, before the first release?

Things I considered: at the end, one would make quite a bit of changes to the API; at the beginning it is time consuming, if the client wants to see first results soon.