User: Don't pull remote users follower count immediately after deactivating.

The other instance doesn't necessarily know that anything changed yet, and it
will be fixed up at the next user pull anyway.

Closes #1369