Search: don't resolve by URL when offset > 0 (add changelog entry) #7978

Open
opencode-fix-bot wants to merge 2 commits from opencode-fix-bot/pleroma:opencode-agent/job-75-g1 into develop
Contributor

Follow-up to !7977, which was missing its changelog entry (per the review comment on it).

Change

Adds changelog.d/search-pagination-url-resolution.fix, the entry required by the check-changelog CI job for PRs targeting develop:

Fix remote objects and accounts being re-fetched and duplicated across search result pages when paginating with offset > 0

This branch is exactly !7977's head (opencode-agent/job-73-g1 @ fe511ccaa104a686132dfe4cebd8bdc9b8e400b4) plus that single changelog commit (361111cf451d90911ba653793656709fe1c6f47d). No code, test, or description changes beyond adding the entry.

Closes #3380 (same as !7977; either PR closes it).

Verification

  • RED: the tools/check-changelog CI logic fails on !7977's base→head diff ("changelog entry not found") — matching the review comment.
  • GREEN: the same check passes with changelog.d/search-pagination-url-resolution.fix added (verified locally against the exact base commit 124f4ece1d2f0786e3774d750cdd84440e744517).
  • Changelog-only change: no .ex/.exs sources touched, so test/format results are unaffected. The Elixir toolchain is not available in the bot environment, so the full suite is left to CI, which runs on this PR.

opencode-agent:job=75:generation=1

Follow-up to !7977, which was missing its changelog entry (per the review comment on it). ## Change Adds `changelog.d/search-pagination-url-resolution.fix`, the entry required by the `check-changelog` CI job for PRs targeting `develop`: > Fix remote objects and accounts being re-fetched and duplicated across search result pages when paginating with offset > 0 This branch is exactly !7977's head (`opencode-agent/job-73-g1` @ `fe511ccaa104a686132dfe4cebd8bdc9b8e400b4`) plus that single changelog commit (`361111cf451d90911ba653793656709fe1c6f47d`). No code, test, or description changes beyond adding the entry. Closes #3380 (same as !7977; either PR closes it). ## Verification - RED: the `tools/check-changelog` CI logic fails on !7977's base→head diff ("changelog entry not found") — matching the review comment. - GREEN: the same check passes with `changelog.d/search-pagination-url-resolution.fix` added (verified locally against the exact base commit `124f4ece1d2f0786e3774d750cdd84440e744517`). - Changelog-only change: no `.ex`/`.exs` sources touched, so test/format results are unaffected. The Elixir toolchain is not available in the bot environment, so the full suite is left to CI, which runs on this PR. opencode-agent:job=75:generation=1
Search: don't resolve by URL when offset > 0
Some checks failed
ci/woodpecker/pr/changelog Pipeline failed
ci/woodpecker/pr/lint Pipeline was successful
ci/woodpecker/pr/unit-testing-elixir-1.15 Pipeline was successful
ci/woodpecker/pr/paradedb-integration Pipeline failed
ci/woodpecker/pr/unit-testing-elixir-1.19 Pipeline was successful
fe511ccaa1
When paginating search results with an offset, the URL query resolution
still ran on every page: DatabaseSearch.maybe_fetch re-fetched the remote
object and prepended it to every page, and User.Search resolved the remote
account again for every page. Skip both when the offset is greater than
zero; resolution results belong to the first page only.

Closes #3380
Add changelog entry for search URL-resolution offset fix
Some checks failed
ci/woodpecker/pr/changelog Pipeline is running
ci/woodpecker/pr/lint Pipeline was successful
ci/woodpecker/pr/unit-testing-elixir-1.19 Pipeline was successful
ci/woodpecker/pr/paradedb-integration Pipeline was canceled
ci/woodpecker/pr/unit-testing-elixir-1.15 Pipeline was canceled
361111cf45
The check-changelog CI job requires a changelog.d entry for changes
targeted at develop; PR !7977 was missing one. Add
changelog.d/search-pagination-url-resolution.fix describing the fix.

opencode-agent:job=75:generation=1

@opencode-fix-bot this broke paradadb, it seems

@opencode-fix-bot this broke paradadb, it seems
Author
Contributor

I dug into this: the red ci/woodpecker/pr/paradedb-integration check is real, but nothing in this PR broke ParadeDB — the job failed during service startup, before a single test ran. cc @lambadalambda

What actually happened (pipeline 596, commit 361111c):

  • The step log ends with:

    + timeout 60 sh -c 'until pg_isready -h paradedb -U postgres -d paradedb; do sleep 1; done'
    paradedb:5432 - no response
    ... (x60 over 60 s)
    Terminated
    + echo 'ParadeDB did not become ready'
    ParadeDB did not become ready
    + exit 1
    

    It aborted before mix ecto.create / mix test ever ran — no test produced any output, and no assertion failed anywhere.

  • The paradedb service container's own log shows why: it was still doing its first-boot initdb + ParadeDB bootstrap. It only logged database system is ready to accept connections at 16:49:49 UTC, ~200 s after the service started (16:46:29 UTC) — i.e. about 13 s after the hard-coded 60 s readiness probe had already given up (16:49:36 UTC).

  • The exact same failure (same signature, same ParadeDB did not become ready) happened on !7977 (pipeline 595, commit fe511cca) — the very commit this PR adds its changelog entry on top of. Versus !7977, this PR's only delta is changelog.d/search-pagination-url-resolution.fix, a file that job never reads, so the failure predates this branch and is unrelated to the changelog entry.

  • The workflow does pass on a runner with a warm image/data cache (pipeline 546 on !7342 succeeded; there the container was up long before the probe ran). On a cold runner the image's init reliably exceeds timeout 60.

Why the diff can't be the cause: with offset: 0 (the default, and what ParadeDBIntegrationTest uses), maybe_fetch/4 behaves exactly as before this change; the only behavioural delta is skipping remote URL resolution when offset > 0, which is the point of the fix. The maybe_fetch arity updates in parade_db.ex / qdrant_search.ex / meilisearch.ex are consistent (lint + credo green), and both unit-test jobs (Elixir 1.15 / 1.19, which include test/pleroma/search/parade_db_test.exs) passed.

Suggested remedies: re-run the pipeline on a warm runner, and/or raise the readiness window in .woodpecker/paradedb-integration.yaml (e.g. timeout 60 → timeout 300 for the pg_isready -h paradedb wait; the 60 s @docker_startup_timeout_ms in test/pleroma/search/parade_db_integration_test.exs only covers the local-docker path). I can send that one-liner as a follow-up PR if you want it.

opencode-agent:job=79:generation=1

I dug into this: the red `ci/woodpecker/pr/paradedb-integration` check is real, but nothing in this PR broke ParadeDB — the job failed during service startup, before a single test ran. cc @lambadalambda **What actually happened** ([pipeline 596](https://ci.pleroma.com/repos/1/pipeline/596/3), commit `361111c`): - The step log ends with: ```text + timeout 60 sh -c 'until pg_isready -h paradedb -U postgres -d paradedb; do sleep 1; done' paradedb:5432 - no response ... (x60 over 60 s) Terminated + echo 'ParadeDB did not become ready' ParadeDB did not become ready + exit 1 ``` It aborted before `mix ecto.create` / `mix test` ever ran — no test produced any output, and no assertion failed anywhere. - The `paradedb` service container's own log shows why: it was still doing its first-boot `initdb` + ParadeDB bootstrap. It only logged `database system is ready to accept connections` at **16:49:49 UTC**, ~200 s after the service started (16:46:29 UTC) — i.e. about **13 s after** the hard-coded 60 s readiness probe had already given up (16:49:36 UTC). - The exact same failure (same signature, same `ParadeDB did not become ready`) happened on **!7977** ([pipeline 595](https://ci.pleroma.com/repos/1/pipeline/595/3), commit `fe511cca`) — the very commit this PR adds its changelog entry on top of. Versus !7977, this PR's only delta is `changelog.d/search-pagination-url-resolution.fix`, a file that job never reads, so the failure predates this branch and is unrelated to the changelog entry. - The workflow does pass on a runner with a warm image/data cache ([pipeline 546](https://ci.pleroma.com/repos/1/pipeline/546/3) on !7342 succeeded; there the container was up long before the probe ran). On a cold runner the image's init reliably exceeds `timeout 60`. **Why the diff can't be the cause:** with `offset: 0` (the default, and what `ParadeDBIntegrationTest` uses), `maybe_fetch/4` behaves exactly as before this change; the only behavioural delta is skipping remote URL resolution when `offset > 0`, which is the point of the fix. The `maybe_fetch` arity updates in `parade_db.ex` / `qdrant_search.ex` / `meilisearch.ex` are consistent (lint + credo green), and both unit-test jobs (Elixir 1.15 / 1.19, which include `test/pleroma/search/parade_db_test.exs`) passed. **Suggested remedies:** re-run the pipeline on a warm runner, and/or raise the readiness window in `.woodpecker/paradedb-integration.yaml` (e.g. `timeout 60` → `timeout 300` for the `pg_isready -h paradedb` wait; the 60 s `@docker_startup_timeout_ms` in `test/pleroma/search/parade_db_integration_test.exs` only covers the local-docker path). I can send that one-liner as a follow-up PR if you want it. opencode-agent:job=79:generation=1
Some checks failed
ci/woodpecker/pr/changelog Pipeline is running
ci/woodpecker/pr/lint Pipeline was successful
ci/woodpecker/pr/unit-testing-elixir-1.19 Pipeline was successful
ci/woodpecker/pr/paradedb-integration Pipeline was canceled
ci/woodpecker/pr/unit-testing-elixir-1.15 Pipeline was canceled
This pull request has changes conflicting with the target branch.
  • lib/pleroma/search/database_search.ex
  • lib/pleroma/search/meilisearch.ex
  • lib/pleroma/search/parade_db.ex
  • lib/pleroma/search/qdrant_search.ex
  • test/pleroma/search/database_search_test.exs
  • test/pleroma/web/mastodon_api/controllers/search_controller_test.exs
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u opencode-agent/job-75-g1:opencode-fix-bot-opencode-agent/job-75-g1
git switch opencode-fix-bot-opencode-agent/job-75-g1

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch develop
git merge --no-ff opencode-fix-bot-opencode-agent/job-75-g1
git switch opencode-fix-bot-opencode-agent/job-75-g1
git rebase develop
git switch develop
git merge --ff-only opencode-fix-bot-opencode-agent/job-75-g1
git switch opencode-fix-bot-opencode-agent/job-75-g1
git rebase develop
git switch develop
git merge --no-ff opencode-fix-bot-opencode-agent/job-75-g1
git switch develop
git merge --squash opencode-fix-bot-opencode-agent/job-75-g1
git switch develop
git merge --ff-only opencode-fix-bot-opencode-agent/job-75-g1
git switch develop
git merge opencode-fix-bot-opencode-agent/job-75-g1
git push origin develop
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
pleroma/pleroma!7978
No description provided.