Support media meta when available
Related: !2607 (merged)
Mastodon has two extra fields on media attachments:
meta
blurhash
On the frontend side, the lack of meta
info makes it challenging to display images and videos in an accessible way. It's mainly due to lack of image dimensions and aspect ratio. Pleroma doesn't support the focal point feature either.
Here's an example of a video shown in legacy Mastodon+Soapbox:
The same video in Pleroma FE:
Here are some examples with multiple images. First, 3 images shown in legacy Mastodon+Soapbox:
The same 3 images in new soapbox-fe, where aspect ratios aren't available:
I believe the meta
param adds significant value to the display of images in the frontend. It's true there are some ways we can work around it in the frontend, but after some experimentation I think we really want to know the image dimensions before we load them (that is, at the same time the status is loaded), or we have to face all sorts of issues with the timeline jumping around while re-rendering things after the image loads.
I understand why Pleroma doesn't have these attachment fields. I think it's extremely worth the trade-off to not store/process thousands of remote images. However I think there are some things we can maybe do to provide these fields anyway:
-
Mastodon sends
meta
andblurhash
data over ActivityPub, and Pleroma doesn't strip it. I think it's reasonable to provide this data in API endpoints if its available. -
Pleroma processes local images, giving the opportunity to attach this data (and federate it over ActivityPub, too).
-
Pleroma can support adding focal points in PUT
/api/v1/media
-
When the media proxy is enabled, it might be possible do some processing on remote images to store
meta
info if it's not already available. Just an idea.
cc @hj