Skip to content

WIP: Fix MediaProxy Invalidation

feld requested to merge fix/mediaproxy-http-invalidation into develop

We added MediaProxy Invalidation via a Script (mostly for Nginx, which can't do cache invalidations via HTTP DELETE or PURGE verbs in their open source version) and HTTP methods. The code was mostly OK, but HTTP version had some problems and we discovered a few edge cases. Here's everything that we expect MediaProxy Invalidation to do:

  • When local posts with attachments are deleted, use Script or Http (PURGE/DELETE) to invalidate cached object. Skip adding entry into Cachex which stops MediaProxy from attempting to re-cache the content if the URL is accessed; local media doesn't go through MediaProxy and would pollute the cache with invalid entries.

  • When federated Deletes arrive, Invalidate the cached object. Also add entry to Cachex to prevent MediaProxy from re-fetching/caching the object in case remote server has not successfully deleted the file yet.

  • When new federated statuses with attachments arrive, attempt to remove them from the MediaProxy cachex: it's possible the user did a delete & redraft and used the same attachment. The URL to the file will be the same, so we want to be able to fetch and cache it.

Unsolved edge case for incoming FEDERATED media:

  1. Make three posts on a remote server with the same attachment
  2. Ensure the three posts federated into your instance
  3. Delete one of the posts, locally as admin or via federated delete
  4. The remaining two posts will inadvertently have their attachments invalidated because they used the same attachment and have the same filename/checksum.

I have been looking into using the attachment_id as a URL parameter as these are unique across posts/statuses.

Edited by feld

Merge request reports