Error: The DELETE statement conflicted with the REFERENCE constraint "FK__aspnet_Me__UserI__15502E78". The conflict occurred in database "YourDBName", table "dbo.aspnet_Membership", column 'UserId'.
Took me a while to find a solution to this across multiple sites and options as the error and possible solutions were rather misleading. Turns out, at least in my case, it was a problem with permissions on the membership database. The user I'm using to connect had access to view the membership details within the database itself, but as part of the aspnet_Users_DeleteUser stored procedure it selects from the sysobjects table. The membership connection user apparently did not have sufficient rights to do that select so the overall delete failed.
The fix for me was to add the user to the aspnet_Membership_FullAccess role for the membership database.