Fix macOS build, process leaks, extension handling, plug, and add CI #16

Merged

lambadalambda merged 14 commits from fix/homebrew-libmagic-paths into develop 2026-05-13 13:47:53 +00:00

Conversation 0 Commits 14 Files changed 20 +360 -121

lambadalambda commented 2026-05-13 06:07:08 +00:00

Owner

Copy link

Summary

Started as a fix for macOS Homebrew libmagic build paths, grew into a broader bugfix pass focused on issues that affect Pleroma in production.

Build & CI

• Use pkg-config to discover libmagic paths (fixes macOS Homebrew build)
• Remove ei_init() call that crashes on macOS with unlimited fd limit
• Redirect C port stderr to /dev/null unless MAJIC_DEBUG is set
• Add Woodpecker CI pipeline (lint + unit tests)

Process Leaks

• Once.perform/3: clean up server process on error (try/after)
• Pool.terminate_worker/3: stop server processes on pool shutdown

Extension.fix

• Preserve directory paths in output (was stripping dir/ prefix)
• Handle uppercase extensions correctly (.JPEG now matches .jpeg)
• No trailing dot when no MIME extension candidates exist
• Reduce cyclomatic complexity below credo threshold

Plug

• Rewrite with recursive transform_uploads — old collect_uploads/put_in_if_exists broke on nested maps/arrays
• Fall back to original upload on majic error instead of raising 500
• Unskip plug test, enable fix_extension option

Server

• Postpone reload/recycle calls in non-available states (was crashing)
• Handle late {:data, _} in recycling state (was crashing)
• Fix :eaccess → :eacces typo in errno map
• Guard terminate/3 against nil port
• Fix Majic.perform once path ignoring :timeout option

Docs

• Fix outdated function references in README
• Update CHANGELOG with all fixes

Tests

• Use plain text magic file instead of compiled .mgc (cross-version compatible)
• 33 → 43 tests, all passing on macOS and Linux (Alpine)

Testing

• mix test --trace passes on macOS (ARM64, Homebrew)
• mix format --check-formatted passes
• mix credo passes (0 issues)
• Docker elixir:1.17-alpine full suite passes (43 tests, 0 failures)

## Summary Started as a fix for macOS Homebrew libmagic build paths, grew into a broader bugfix pass focused on issues that affect Pleroma in production. ### Build & CI - Use `pkg-config` to discover libmagic paths (fixes macOS Homebrew build) - Remove `ei_init()` call that crashes on macOS with unlimited fd limit - Redirect C port stderr to `/dev/null` unless `MAJIC_DEBUG` is set - Add Woodpecker CI pipeline (lint + unit tests) ### Process Leaks - `Once.perform/3`: clean up server process on error (try/after) - `Pool.terminate_worker/3`: stop server processes on pool shutdown ### Extension.fix - Preserve directory paths in output (was stripping `dir/` prefix) - Handle uppercase extensions correctly (`.JPEG` now matches `.jpeg`) - No trailing dot when no MIME extension candidates exist - Reduce cyclomatic complexity below credo threshold ### Plug - Rewrite with recursive `transform_uploads` — old `collect_uploads`/`put_in_if_exists` broke on nested maps/arrays - Fall back to original upload on majic error instead of raising 500 - Unskip plug test, enable `fix_extension` option ### Server - Postpone `reload`/`recycle` calls in non-available states (was crashing) - Handle late `{:data, _}` in recycling state (was crashing) - Fix `:eaccess` → `:eacces` typo in errno map - Guard `terminate/3` against nil port - Fix `Majic.perform` once path ignoring `:timeout` option ### Docs - Fix outdated function references in README - Update CHANGELOG with all fixes ### Tests - Use plain text magic file instead of compiled `.mgc` (cross-version compatible) - 33 → 43 tests, all passing on macOS and Linux (Alpine) ## Testing - `mix test --trace` passes on macOS (ARM64, Homebrew) - `mix format --check-formatted` passes - `mix credo` passes (0 issues) - Docker `elixir:1.17-alpine` full suite passes (43 tests, 0 failures)

lambadalambda added 1 commit 2026-05-13 06:07:08 +00:00

Use pkg-config to discover libmagic paths 3ee9e73ddb

lambadalambda added 1 commit 2026-05-13 06:52:42 +00:00

Fix tests on macOS with Homebrew libmagic ... 16cb4f9f60

- Remove ei_init() call that fails when sysconf(_SC_OPEN_MAX) returns
  LONG_MAX (unlimited fd limit on macOS), causing int overflow in
  Erlang's init_socket_info(). The port only uses ei for
  encoding/decoding, not node connections.
- Redirect port stderr to /dev/null unless MAJIC_DEBUG is set, to
  suppress libmagic warnings in test output.
- Compile test .mgc with matching file binary from Homebrew file-formula
  to avoid version mismatch with the linked libmagic.
- Increase port test receive timeout to accommodate slower port startup.
- Broaden .gitignore to cover /priv/ and *.mgc.

lambadalambda added 1 commit 2026-05-13 07:56:52 +00:00

Fix review findings: leaks, timeouts, C safety, extensions, plug ... 1839c8e9dd

- Fix Once.perform/3 server process leak on error (try/after)
- Fix Pool.terminate_worker/3 not stopping worker servers
- Fix server timeouts using bare integers instead of :state_timeout
- Postpone reload/recycle in non-available states
- Guard Server.terminate/3 against nil port
- Validate Server.perform/3 input, return {:error, :badarg}
- Fix :eaccess -> :eacces errno mapping
- Fix C atom buffer too small (128 -> MAXATOMLEN)
- Fix C fdseek mixing read(2) and getchar()
- Fix C header read robustness (read_exact, unsigned char casts)
- Check C ei_get_type return values
- Free C ei_x_buff on reload early return
- Add NULL checks for magic_file/magic_buffer return values
- Fix Extension.fix/3 stripping directory paths
- Fix Extension.fix/3 mishandling uppercase extensions
- Fix Extension.fix/3 trailing dot with no candidates
- Fix Majic.perform once:true ignoring :timeout option
- Rewrite Majic.Plug with recursive transformation
- Enable previously skipped Plug test
- Add new tests for process leaks, invalid input, oversized packets

lambadalambda added 6 commits 2026-05-13 08:04:10 +00:00

Fix Extension.fix/3 duplicating directory paths with append: true ... e359afbc98

Path.join('.', x) prepends './' so use dir != '.' guard.
Use Path.basename(name) for append branches to avoid including
directory in the filename portion being appended to.

Handle port exit/closed in processing state and late data in recycling ... 6bffff1480

Processing state now handles :exit_status and :closed port messages,
replying with {:error, :port_exit} / {:error, :port_closed} and
transitioning to :starting. Recycling state now postpones late
{:data, _} messages instead of crashing.

Fix Pool.perform/3 raising on checkout timeout ... 6f8b8fbe6a

Catch :exit {:timeout, _} from NimblePool.checkout!/3 and return
{:error, :timeout} instead of crashing the caller.

Add EISDIR (errno 21) to errno map for completeness ... fa01e6f518

Libmagic handles directories gracefully (returns inode/directory)
but some platforms may still return errno 21, so map it defensively.

Fix C: NULL guard in handle_magic_error and ei_get_type in process_command_bytes ... df6c417cb7

magic_error() can return NULL; guard against strlen(NULL) by encoding
'unknown' atom when error message is unavailable. Also fix
process_command_bytes to check ei_get_type return value instead of
using EI_ENSURE which would exit the port process.

Fix Majic.Plug processing uploads twice via params + body_params ... be0dd96941

Only transform body_params, then merge with query_params to build
params. This avoids calling Majic.perform on the same upload file
twice (once via body_params, once via the merged params map).

lambadalambda force-pushed fix/homebrew-libmagic-paths from be0dd96941 to fc689178ab 2026-05-13 08:33:27 +00:00 Compare

lambadalambda added 2 commits 2026-05-13 08:59:19 +00:00

Postpone reload/recycle in non-available states, handle late data in recycling ... d66006f5f5

Calling reload or recycle during processing/starting/loading/recycling
previously crashed the server. Now these calls are postponed until the
server returns to :available. Also discard late {:data, _} messages in
recycling state instead of crashing. Fix :eaccess -> :eacces typo and
add is_port guard in terminate/3.

Fix Plug raising on majic error instead of falling back ... 331f7afef4

When Majic.perform fails (e.g. temp file deleted, permission denied,
pool timeout), return the original upload unchanged instead of raising
a 500 error.

lambadalambda added 1 commit 2026-05-13 11:05:29 +00:00

Add Woodpecker CI pipeline for lint and unit tests ... b634668f08

Lint pipeline: mix format --check-formatted, credo
Unit test pipeline: mix test --trace --preload-modules
Both use elixir:1.17-alpine with file-dev for libmagic headers.

lambadalambda added 1 commit 2026-05-13 12:14:26 +00:00

Fix outdated references in README and update CHANGELOG ... dc8f9d47dc

README: fix function names (Once.perform/3, start_option/0),
remove references to nonexistent Majic.Helpers.perform_once/2,
clarify Plug fix_extension is opt-in, fix recycle_threshold typo.

CHANGELOG: remove nonexistent Majic.compile/2, add Fixed section
with all bug fixes, add macOS/Homebrew build support and Woodpecker CI.

lambadalambda added 1 commit 2026-05-13 12:16:21 +00:00

Fix format and credo issues for CI ... a8cbe3c4a1

- Run mix format on all files
- Reduce Extension.do_fix cyclomatic complexity below 9
  by extracting fix_existing_ext and fix_no_ext helpers
- Remove unused has_candidates? variable

lambadalambda added 1 commit 2026-05-13 12:24:17 +00:00

Fix CI: add file package, use plain text magic file instead of .mgc ... aac3cb655f

- Add 'file' to apk packages (needed for C port, not just file-dev)
- Clean stale macOS binary before compile on CI
- Use plain text magic file directly instead of compiled .mgc
  (.mgc format is version-dependent and breaks across different
  libmagic versions)

lambadalambda changed title from Use pkg-config to discover libmagic paths to Fix macOS build, process leaks, extension handling, plug, and add CI 2026-05-13 12:26:00 +00:00

lambadalambda added 1 commit 2026-05-13 12:28:10 +00:00

Fix README: reference Majic.perform with once: true, not Once.perform directly 8742cc7f70

lambadalambda force-pushed fix/homebrew-libmagic-paths from 8742cc7f70 to dbd654d0e6 2026-05-13 12:30:10 +00:00 Compare

lambadalambda merged commit 1f235da0fa into develop 2026-05-13 13:47:53 +00:00

lambadalambda referenced this pull request from a commit 2026-05-13 13:47:53 +00:00

Merge pull request 'Fix macOS build, process leaks, extension handling, plug, and add CI' (#16) from fix/homebrew-libmagic-paths into develop

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

pleroma-elixir-libraries/majic!16

No description provided.