Skip to content

Draft: prune old transient activities and tombstone objects

faried nawaz requested to merge faried/pleroma:feature/prune-deletes into develop

This is marked as a draft PR; someone (else) should verify that it works properly.

This creates a mix task + Oban worker to periodically prune all old Delete, Undo, and Remove activities, and Tombstone objects from the database. The cut-off date is the number of days defined in the config with :remote_post_retention_days.

Against a copy of my production database, before:

pleroma_dev=> select data->>'type', count(data->>'type') as count from activities where inserted_at < now() - interval '90 days' group by data->>'type' order by count desc;
   ?column?    |  count  
---------------+---------
 Create        | 2757638
 Delete        |  477695
 Like          |  389895
 Announce      |  326735
 Update        |   81100
 Undo          |   56268
 EmojiReact    |   11185
 Accept        |   10887
 Follow        |     756
 Add           |     743
 Remove        |     377
 EmojiReaction |     104
 Move          |      41
 Reject        |      30
 Block         |      14
 Flag          |       1
(16 rows)

After:

   ?column?    |  count  
---------------+---------
 Create        | 2757649
 Like          |  389895
 Announce      |  326740
 Update        |   81101
 EmojiReact    |   11185
 Accept        |   10887
 Follow        |     756
 Add           |     743
 Delete        |     306
 EmojiReaction |     104
 Move          |      41
 Reject        |      30
 Block         |      14
 Undo          |       9
 Flag          |       1
(15 rows)

This is based on https://akkoma.dev/AkkomaGang/akkoma/pulls/327 with one difference: the Akkoma PR also discards incoming Undo, Delete, and Remove activities in its Pleroma.Web.ActivityPub.ActivityPub persist function:

  @unpersisted_activity_types ~w[Undo Delete Remove]
  @impl true
  def persist(%{"type" => type} = object, [local: false] = meta)
      when type in @unpersisted_activity_types do
    {recipients, _, _} = get_recipients(object)

    unpersisted = %Activity{
      data: object,
      local: false,
      recipients: recipients,
      actor: object["actor"]
    }

    {:ok, unpersisted, meta}
  end

A later PR discards Accept and Remove activities as well.

Checklist

  • Adding a changelog: In the changelog.d directory, create a file named <code>.<type>.

Merge request reports