SPFieldValidationException not handled as expected as custom field validation

I try to add custom validation by overriding GetValidatedString in a custom field that inherits SPFieldChoice and throwing a SPFieldValidationException as described here.

This does not work, the system behaves just like it would have been any other exception and displays an error page instead of the expected form with the validation message for the specific field.

I tried the same on another custom field that inherits SPFieldText and that works. Are there constraints on which field types the validation logic based on SPFieldValidationException works and not? Is there something else that could be wrong?