Makes it so that a user can remove an existing follower.
Corresponding backend MR: pleroma!3647 (merged)
