pleroma/pleroma
4
6
Fork 12
Code Issues 720 Pull requests 83 Projects Releases 1 Packages Wiki Activity

paradedb-search-backend #7849

Open
lambadalambda wants to merge 20 commits from paradedb-search-backend into develop
pull from: paradedb-search-backend
merge into: pleroma:develop
Conversation 3 Commits 20 Files changed 20 +1141 -50
lambadalambda commented 2026-03-05 14:46:50 +00:00
Owner
Copy link

Checklist

  • Adding a changelog: In the changelog.d directory, create a file named <code>.<type>.
### Checklist - [ ] Adding a changelog: In the `changelog.d` directory, create a file named `<code>.<type>`. <!-- `<code>` can be anything, but we recommend using a more or less unique identifier to avoid collisions, such as the branch name. `<type>` can be `add`, `change`, `remove`, `fix`, `security` or `skip`. `skip` is only used if there is no user-visible change in the PR (for example, only editing comments in the code). Otherwise, choose a type that corresponds to your change. In the file, write the changelog entry. For example, if a PR adds group functionality, we can create a file named `group.add` and write `Add group functionality` in it. If one changelog entry is not enough, you may add more. But that might mean you can split it into two PRs. Only use more than one changelog entry if you really need to (for example, when one change in the code fix two different bugs, or when refactoring). --> 
Introduce an optional search backend that indexes public/unlisted Notes into a separate ParadeDB (pg_search) Postgres and queries it for status search results.
Add a skipped-by-default integration test that boots a ParadeDB container, creates the BM25 index, indexes posts, and verifies search results against the real backend.
Remove warnings that break --warnings-as-errors, add TCP readiness check, and start the ParadeDB Repo with a small pool to reduce connection churn while booting the dockerized backend.
Use uuid ids in the ParadeDB documents table to satisfy BM25 key_field requirements and adjust indexing/search to dump FlakeIds consistently. Also fix the ParadeDB backend unit test to avoid invalid pinning on struct fields.
Re-apply public Create visibility checks when resolving backend hits, improve UUID/FlakeId id parsing, and add tests for poisoned hits and UUID-string ids. Also tighten the docker integration test with localhost-only binding and config restoration.
Make ParadeDB search typo-tolerant via configurable fuzzy distance and document the new setting. Add coverage for fuzzy query SQL and id parsing, and keep integration test stability under connection load.
Cast the bound search parameter through text before  so Postgrex can handle the prepared statement type inference. Add matching test assertion updates.
Add ParadeDB fuzzy integration coverage
All checks were successful
ci/woodpecker/pr/woodpecker Pipeline was successful
Details
cae3c9a980
 
Extend the docker-backed ParadeDB integration test to verify typo-tolerant status search with fuzzy_distance enabled, covering prepared-parameter fuzzy casts against a real backend.
Merge branch 'develop' into paradedb-search-backend
All checks were successful
ci/woodpecker/pr/woodpecker Pipeline was successful
Details
865ddfdce0
feld commented 2026-03-07 05:29:29 +00:00
Owner
Copy link

mix tasks don't work unless you update lib/mix/pleroma.ex with the same logic on when to start the new Repo

edit: done

mix tasks don't work unless you update lib/mix/pleroma.ex with the same logic on when to start the new Repo edit: done
feld commented 2026-03-07 05:43:24 +00:00
Owner
Copy link

I don't know that we should default to a pool_size of 10

It would be nice if we moved some of the mix task migration logic out of the mix task and into the backend so we can run it from an attached console easily to do the migration online

I don't know that we should default to a pool_size of 10 It would be nice if we moved some of the mix task migration logic out of the mix task and into the backend so we can run it from an attached console easily to do the migration online
feld commented 2026-03-25 19:32:59 +00:00
Owner
Copy link

Can we include the post subject and the media/attachment descriptions in the index? If we make them separate columns we have to rebuild the entire search index on our test servers, but it might give us more flexibility in the future. Otherwise we have to just concatenate that text into the post content I guess

Can we include the post subject and the media/attachment descriptions in the index? If we make them separate columns we have to rebuild the entire search index on our test servers, but it might give us more flexibility in the future. Otherwise we have to just concatenate that text into the post content I guess
Fix warning
Some checks failed
ci/woodpecker/pr/woodpecker Pipeline failed
Details
3776f13f20
Merge branch 'develop' into paradedb-search-backend
Some checks failed
ci/woodpecker/pr/changelog Pipeline was successful
Details
ci/woodpecker/pr/lint Pipeline failed
Details
ci/woodpecker/pr/unit-testing-elixir-1.18 unknown status
Details
ci/woodpecker/pr/unit-testing-elixir-1.15 unknown status
Details
7e97216b52
Merge branch 'develop' into paradedb-search-backend
Some checks failed
ci/woodpecker/pr/changelog Pipeline was successful
Details
ci/woodpecker/pr/lint Pipeline failed
Details
ci/woodpecker/pr/unit-testing-elixir-1.15 unknown status
Details
ci/woodpecker/pr/unit-testing-elixir-1.18 unknown status
Details
be2a5056a3
Merge branch 'develop' into paradedb-search-backend
Some checks failed
ci/woodpecker/pr/changelog Pipeline was successful
Details
ci/woodpecker/pr/lint Pipeline failed
Details
ci/woodpecker/pr/unit-testing-elixir-1.15 unknown status
Details
ci/woodpecker/pr/unit-testing-elixir-1.18 unknown status
Details
ci/woodpecker/pr/docker-armv7 Pipeline was successful
Details
ac05f65fbb
Fix ParadeDB search indexing
Some checks failed
ci/woodpecker/pr/changelog Pipeline was successful
Details
ci/woodpecker/pr/lint Pipeline failed
Details
ci/woodpecker/pr/unit-testing-elixir-1.18 unknown status
Details
ci/woodpecker/pr/unit-testing-elixir-1.15 unknown status
Details
ci/woodpecker/pr/docker-armv7 Pipeline was successful
Details
cd28c2089c
Some checks failed
ci/woodpecker/pr/changelog Pipeline was successful
Details
ci/woodpecker/pr/lint Pipeline failed
Details
ci/woodpecker/pr/unit-testing-elixir-1.18 unknown status
Details
ci/woodpecker/pr/unit-testing-elixir-1.15 unknown status
Details
ci/woodpecker/pr/docker-armv7 Pipeline was successful
Details
This pull request can be merged automatically.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin paradedb-search-backend:paradedb-search-backend
git switch paradedb-search-backend

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch develop
git merge --no-ff paradedb-search-backend
git switch paradedb-search-backend
git rebase develop
git switch develop
git merge --ff-only paradedb-search-backend
git switch paradedb-search-backend
git rebase develop
git switch develop
git merge --no-ff paradedb-search-backend
git switch develop
git merge --squash paradedb-search-backend
git switch develop
git merge --ff-only paradedb-search-backend
git switch develop
git merge paradedb-search-backend
git push origin develop
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
AP C2S

   
AdminAPI

   
Breaking API
Affects clients

  
Breaks existing config

   
Check if still valid

   
Data Protection

   
Do not merge

   
Document in the changelog

   
Doing

   
Feature Request / Enhancement

   
MastoAPI

   
NL2
Improved Mobile Experience

  
NL4
Improved instance customization and QoL options

  
OStatus

   
OTP Releases

   
Refactor

   
Security

   
To Do

   
TwitterAPI

   
bug

   
complicated

   
confirmed

   
deps

   
discussion

   
documentation

   
easy

   
feature: to be merged in stable

   
fix: to be merged in stable

   
incident
Denotes a disruption to IT services and the associated issues require immediate attention

  
needs to be backported to maint

   
needs-changes

   
needs-info

   
needs-review

   
performance

   
regression

   
unclear
No labels
AP C2S
AdminAPI
Breaking API
Breaks existing config
Check if still valid
Data Protection
Do not merge
Document in the changelog
Doing
Feature Request / Enhancement
MastoAPI
NL2
NL4
OStatus
OTP Releases
Refactor
Security
To Do
TwitterAPI
bug
complicated
confirmed
deps
discussion
documentation
easy
feature: to be merged in stable
fix: to be merged in stable
incident
needs to be backported to maint
needs-changes
needs-info
needs-review
performance
regression
unclear
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
pleroma/pleroma!7849
No description provided.