Uploads: replace exiftool with the built-in exif_stripper #8028

Merged

lambadalambda merged 10 commits from lambadalambda/pleroma:feature/exif-stripper into develop 2026-10-31 07:48:01 +00:00

Conversation 4 Commits 10 Files changed 44 +509 -372

lambadalambda commented 2026-10-27 14:15:07 +00:00

Owner

Copy link

Replaces the runtime exiftool dependency of the upload filters with the pure-Elixir exif_stripper 0.1.3.

• New filters Exif.StripLocation (lossless GPS stripping for JPEG, PNG, WebP, HEIC/AVIF and TIFF) and Exif.ReadDescription.
• Security: the old filter skipped HEIC and WebP, so iPhone HEIC uploads kept their GPS location. Now they are stripped (tested with a real gridded HEIC as iPhones write it).
• The Exiftool, Exiftool.StripLocation and Exiftool.ReadDescription names keep working as deprecated aliases, with a startup rewrite and a ConfigDB migration (which leaves configs without :filters alone).
• exiftool is gone from the startup requirements, the instance generator, the install docs and the Docker images. The CI images keep it, because the tests use it to check results.

A file with malformed metadata now fails the upload instead of being skipped. This is a rebuilt version of the old exif branch on current develop.

With 0.1.3, GPS data in XMP (including DJI drone positions) is stripped too, TIFF descriptions are read, and uploads are no longer read fully into memory (about 1 MB extra for a 20 MB image).

Replaces the runtime `exiftool` dependency of the upload filters with the pure-Elixir [exif_stripper](https://hex.pm/packages/exif_stripper) 0.1.3. - New filters `Exif.StripLocation` (lossless GPS stripping for JPEG, PNG, WebP, HEIC/AVIF and TIFF) and `Exif.ReadDescription`. - **Security:** the old filter skipped HEIC and WebP, so iPhone HEIC uploads kept their GPS location. Now they are stripped (tested with a real gridded HEIC as iPhones write it). - The `Exiftool`, `Exiftool.StripLocation` and `Exiftool.ReadDescription` names keep working as deprecated aliases, with a startup rewrite and a ConfigDB migration (which leaves configs without `:filters` alone). - exiftool is gone from the startup requirements, the instance generator, the install docs and the Docker images. The CI images keep it, because the tests use it to check results. A file with malformed metadata now fails the upload instead of being skipped. This is a rebuilt version of the old `exif` branch on current develop. With 0.1.3, GPS data in XMP (including DJI drone positions) is stripped too, TIFF descriptions are read, and uploads are no longer read fully into memory (about 1 MB extra for a 20 MB image).

lambadalambda added 9 commits 2026-10-27 14:15:07 +00:00

build: add exif_stripper dependency 51800008eb

feat(upload): add Exif.StripLocation and Exif.ReadDescription filters ... addbf23546

Pure-Elixir filters based on exif_stripper. StripLocation also handles
WebP and HEIC/AVIF, which the exiftool filter skipped.

refactor(upload): make Exiftool filters deprecated aliases of Exif filters ... dd337b92f1

Old configs keep working and now also strip GPS from WebP and HEIC.
The deprecation warning names the replacement for each old filter, and
exiftool is no longer a startup requirement.

feat(instance): enable Exif upload filters without exiftool check df280b1d6b

docs: drop exiftool from install docs and Docker images 114d683683

feat(config): migrate Exiftool upload filters in ConfigDB to Exif names bd6602f4ce

build: bump exif_stripper to 0.1.2 ... d1deb9da45

0.1.2 handles gridded HEIC photos (as iPhones write them), PNG files
with data after IEND and TIFF files, and bounds HEIC parsing by the
input size. Add tests for these files.

fix(migration): leave ConfigDB upload configs without filters alone ... ce8b2276c2

Keyword.update/4 added filters: [] to a Pleroma.Upload ConfigDB entry
without a :filters key, which would hide the filters from the config
file, GPS stripping included. Only rewrite lists that change.

docs(upload): mention TIFF and the XMP limit in the Exif filter docs e083a299c3

lambadalambda commented 2026-10-27 14:15:15 +00:00

Author

Owner

Copy link

/oc review

/oc review

👀 1

opencode-fix-bot reviewed 2026-10-27 14:43:37 +00:00

opencode-fix-bot left a comment

Copy link

opencode-agent:job=129:generation=1

Review: replace exiftool with exif_stripper

Merge-readiness: 4/5 — one P2 finding, otherwise solid.

What I checked: the full base-to-head diff (42 files), the pinned exif_stripper 0.1.2 sources (GpsStripper + Description), the Pleroma.Upload.Filter pipeline, the ConfigDB migration, the deprecation rewrite, and parity against the old exiftool invocations — including an empirical check that the old -gps:all= invocation also left XMP-embedded GPS in place, so the new library preserving XMP is parity, not a regression.

P2 findings

P2: Exif.ReadDescription silently drops TIFF descriptions (regression vs exiftool)

• Trigger: any TIFF upload carrying an EXIF ImageDescription, with the ReadDescription filter enabled. Content type image/tiff passes the "image" <> _ gate in lib/pleroma/upload/filter/exif/read_description.ex, so the file reaches the library.
• Impact: the old filter shelled out to exiftool for any tempfile, so TIFF descriptions were returned and used to prefill the media description. ExifStripper.Description.read_binary/2 only matches JPEG, PNG, WebP, GIF and ISO-BMFF magic — TIFF (II*\0/MM\0*) falls through to the catch-all [], returns nil, and the filter reports {:ok, :noop}. TIFF uploads lose description prefill with no log or other signal. Note StripLocation does handle TIFF, so only the read side is affected.
• Smallest correction: add a TIFF branch to the description reader (reusing the same exif_ascii_tag lookup as the other containers) in exif_stripper, or document that TIFF descriptions are no longer read.

No P0/P1 findings. The fail-closed behavior on malformed metadata is disclosed in the PR description and is a deliberate trade-off; the deprecated alias modules, the startup rewrite (with dedup), the ConfigDB migration (leaves filter-less configs alone), and the byte-identical :noop path all check out. SVG/GIF resolve to :noop via the library catch-all, and PNG text-chunk removal matches the old -png:all= behavior.

opencode-agent:job=129:generation=1 ## Review: replace exiftool with exif_stripper Merge-readiness: 4/5 — one P2 finding, otherwise solid. What I checked: the full base-to-head diff (42 files), the pinned `exif_stripper` 0.1.2 sources (`GpsStripper` + `Description`), the `Pleroma.Upload.Filter` pipeline, the ConfigDB migration, the deprecation rewrite, and parity against the old exiftool invocations — including an empirical check that the old `-gps:all=` invocation also left XMP-embedded GPS in place, so the new library preserving XMP is parity, not a regression. ### P2 findings **P2: `Exif.ReadDescription` silently drops TIFF descriptions (regression vs exiftool)** - Trigger: any TIFF upload carrying an EXIF ImageDescription, with the ReadDescription filter enabled. Content type `image/tiff` passes the `"image" <> _` gate in `lib/pleroma/upload/filter/exif/read_description.ex`, so the file reaches the library. - Impact: the old filter shelled out to exiftool for any tempfile, so TIFF descriptions were returned and used to prefill the media description. `ExifStripper.Description.read_binary/2` only matches JPEG, PNG, WebP, GIF and ISO-BMFF magic — TIFF (`II*\0`/`MM\0*`) falls through to the catch-all `[]`, returns `nil`, and the filter reports `{:ok, :noop}`. TIFF uploads lose description prefill with no log or other signal. Note `StripLocation` *does* handle TIFF, so only the read side is affected. - Smallest correction: add a TIFF branch to the description reader (reusing the same `exif_ascii_tag` lookup as the other containers) in exif_stripper, or document that TIFF descriptions are no longer read. No P0/P1 findings. The fail-closed behavior on malformed metadata is disclosed in the PR description and is a deliberate trade-off; the deprecated alias modules, the startup rewrite (with dedup), the ConfigDB migration (leaves filter-less configs alone), and the byte-identical `:noop` path all check out. SVG/GIF resolve to `:noop` via the library catch-all, and PNG text-chunk removal matches the old `-png:all=` behavior.

lambadalambda added 1 commit 2026-10-28 07:36:01 +00:00

build: bump exif_stripper to 0.1.3 ... 55bf6b851d

0.1.3 strips GPS data from XMP (including DJI drone positions), reads
TIFF descriptions, and no longer reads whole images into memory.

lambadalambda commented 2026-10-28 07:36:05 +00:00

Author

Owner

Copy link

/oc review

/oc review

👀 1

opencode-fix-bot reviewed 2026-10-28 07:57:14 +00:00

opencode-fix-bot left a comment

Copy link

opencode-agent:job=135:generation=1

Review: replace exiftool with exif_stripper

Merge-readiness: 5/5 — no findings. Production-ready as reviewed.

What I checked: the complete PR diff (44 files, merge-base to head), the pinned exif_stripper 0.1.3 sources (GpsStripper + Description), the Pleroma.Upload.Filter pipeline semantics, the ConfigDB migration, the deprecation rewrite, and parity against the old exiftool invocations. Empirically: compiled the tree cleanly and ran the full exif-related suite — the 23 Exif.*/Exiftool filter tests plus the migration, deprecation-warning, and instance-generator tests (28 tests) — all pass. I also verified a TIFF ImageDescription round-trips through the new read path, confirming the P2 from the previous review (TIFF descriptions dropped under 0.1.2) is fixed by the 0.1.3 bump and is not re-raised.

Spot-checks that held up:

• StripLocation strips GPS from JPEG, PNG, WebP, TIFF, and both HEIC variants (gridded included), patches same-file inputs in place only after the full patch plan is computed (with an overlapping-patch safety net), and uses temp-file + rename with mode preservation for rebuilds (PNG). Other EXIF data is preserved; no re-encoding.
• ReadDescription enforces the description limit and blank-value rules in the library, returns :noop (not a nil :filtered) when nothing is found, and never raises on missing files — the pipeline treats it identically to the old flow.
• Deprecated Exiftool* names delegate to the new filters (the bare Exiftool alias is new and covers configs the old startup check already referenced); the startup rewrite dedups collisions and the migration leaves filter-less configs untouched.
• Fail-closed behavior on malformed metadata is disclosed in the PR description and is a deliberate trade-off, not a defect.
• Docs, install guides, Docker images, changelog entries, and CI (which keeps exiftool as the test oracle) are consistent with the change.

No P0/P1/P2 findings. The previously reported TIFF-description gap is addressed in the current code.

opencode-agent:job=135:generation=1 ## Review: replace exiftool with exif_stripper Merge-readiness: 5/5 — no findings. Production-ready as reviewed. What I checked: the complete PR diff (44 files, merge-base to head), the pinned `exif_stripper` 0.1.3 sources (`GpsStripper` + `Description`), the `Pleroma.Upload.Filter` pipeline semantics, the ConfigDB migration, the deprecation rewrite, and parity against the old exiftool invocations. Empirically: compiled the tree cleanly and ran the full exif-related suite — the 23 `Exif.*`/`Exiftool` filter tests plus the migration, deprecation-warning, and instance-generator tests (28 tests) — all pass. I also verified a TIFF `ImageDescription` round-trips through the new read path, confirming the P2 from the previous review (TIFF descriptions dropped under 0.1.2) is fixed by the 0.1.3 bump and is not re-raised. Spot-checks that held up: - `StripLocation` strips GPS from JPEG, PNG, WebP, TIFF, and both HEIC variants (gridded included), patches same-file inputs in place only after the full patch plan is computed (with an overlapping-patch safety net), and uses temp-file + rename with mode preservation for rebuilds (PNG). Other EXIF data is preserved; no re-encoding. - `ReadDescription` enforces the description limit and blank-value rules in the library, returns `:noop` (not a nil `:filtered`) when nothing is found, and never raises on missing files — the pipeline treats it identically to the old flow. - Deprecated `Exiftool*` names delegate to the new filters (the bare `Exiftool` alias is new and covers configs the old startup check already referenced); the startup rewrite dedups collisions and the migration leaves filter-less configs untouched. - Fail-closed behavior on malformed metadata is disclosed in the PR description and is a deliberate trade-off, not a defect. - Docs, install guides, Docker images, changelog entries, and CI (which keeps exiftool as the test oracle) are consistent with the change. No P0/P1/P2 findings. The previously reported TIFF-description gap is addressed in the current code.

lambadalambda merged commit 3e9dc477c9 into develop 2026-10-31 07:48:01 +00:00

lambadalambda referenced this pull request from a commit 2026-10-31 07:48:01 +00:00

Merge pull request 'Uploads: replace exiftool with the built-in exif_stripper' (#8028) from lambadalambda/pleroma:feature/exif-stripper into develop

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

AP C2S

  

AdminAPI

  

Breaking API

Affects clients

  

Breaks existing config

  

Check if still valid

  

Data Protection

  

Do not merge

  

Document in the changelog

  

Doing

  

Feature Request / Enhancement

  

MastoAPI

  

NL2

Improved Mobile Experience

  

NL4

Improved instance customization and QoL options

  

OStatus

  

OTP Releases

  

Refactor

  

Security

  

To Do

  

TwitterAPI

  

bug

  

complicated

  

confirmed

  

deps

  

discussion

  

documentation

  

easy

  

feature: to be merged in stable

  

fix: to be merged in stable

  

incident

Denotes a disruption to IT services and the associated issues require immediate attention

  

needs to be backported to maint

  

needs-changes

  

needs-info

  

needs-review

  

performance

  

regression

  

unclear

No labels

AP C2S

AdminAPI

Breaking API

Breaks existing config

Check if still valid

Data Protection

Do not merge

Document in the changelog

Doing

Feature Request / Enhancement

MastoAPI

NL2

NL4

OStatus

OTP Releases

Refactor

Security

To Do

TwitterAPI

bug

complicated

confirmed

deps

discussion

documentation

easy

feature: to be merged in stable

fix: to be merged in stable

incident

needs to be backported to maint

needs-changes

needs-info

needs-review

performance

regression

unclear

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

pleroma/pleroma!8028

No description provided.