Should I worry about exposing my server’s IP address?

I currently use Cloudflare for DNS, and they advertise that they will hide the IP address of my server if I proxy through them. Should I worry about this since the IP points to a dedicated server (not a home network or anything)?
Are there any security concerns or general risks associated with exposing a server’s IP address?