pleroma/pleroma
4
6
Fork 12
Code Issues 719 Pull requests 82 Projects Releases 1 Packages Wiki Activity

Deletion: Handle the case of pruned objects. #5885

Closed
lambadalambda wants to merge 0 commits from gitlab-mr-iid-2507 into develop
pull from: gitlab-mr-iid-2507
merge into: pleroma:develop
Conversation 3 Commits - Files changed -
lambadalambda commented 2020-05-11 13:09:06 +00:00
Owner
Copy link

Closes #1204

I'd recommend removing pruning completely, it makes ensuring a consistent database state much harder, for little gain. We should use other mechanisms to prevent extensive space usage over time.

Closes #1204 I'd recommend removing pruning completely, it makes ensuring a consistent database state much harder, for little gain. We should use other mechanisms to prevent extensive space usage over time.
lambadalambda commented 2020-05-11 13:10:03 +00:00
Author
Owner
Copy link

Also, while this should probably be merged into stable, this part changed so much it's hard to port. Don't really know what to do about this except writing a separate fix for stable.

Also, while this should probably be merged into stable, this part changed so much it's hard to port. Don't really know what to do about this except writing a separate fix for stable.
rinpatch commented 2020-05-11 13:13:08 +00:00
Member
Copy link

"little gain" is much arguable. However I get the concern, I think we should just support backdating activities, which would allow us to remove the creates with objects as if they were never there

"little gain" is much arguable. However I get the concern, I think we should just support backdating activities, which would allow us to remove the creates with objects as if they were never there
lambadalambda commented 2020-05-11 13:25:49 +00:00
Author
Owner
Copy link

Yeah, we should remove the create activities with it, then it wouldn't be much of an issue.

I absolutely see the concern for dbs getting too large, but pruning objects is not the right approach, especially if it complicates the database state this much. Pruning can't even save much space, it only deletes objects, which are only a small part of overall remote data.

If some remote users posts something, we get a create activity and an object. Now 20 remote people we follow click like on that, we now have 20 more activities. Then pruning happens some time and instead of 22 database objects of similar size we now have 21. That's what I mean by little gain. Reducing disk usage is necessary, but pruning (in the current way) is not the right approach.

We also have two different pruning mechanisms, one is Object.prune (which as far as I can tell isn't used at all) and the one in the mix task. I think a task that would just delete all remote activities older than a cutoff date would make more sense.

Yeah, we should remove the create activities with it, then it wouldn't be much of an issue. I absolutely see the concern for dbs getting too large, but pruning objects is not the right approach, especially if it complicates the database state this much. Pruning can't even save much space, it only deletes objects, which are only a small part of overall remote data. If some remote users posts something, we get a create activity and an object. Now 20 remote people we follow click like on that, we now have 20 more activities. Then pruning happens some time and instead of 22 database objects of similar size we now have 21. That's what I mean by little gain. Reducing disk usage is necessary, but pruning (in the current way) is not the right approach. We also have two different pruning mechanisms, one is `Object.prune` (which as far as I can tell isn't used at all) and the one in the mix task. I think a task that would just delete all remote activities older than a cutoff date would make more sense.

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
AP C2S

   
AdminAPI

   
Breaking API
Affects clients

  
Breaks existing config

   
Check if still valid

   
Data Protection

   
Do not merge

   
Document in the changelog

   
Doing

   
Feature Request / Enhancement

   
MastoAPI

   
NL2
Improved Mobile Experience

  
NL4
Improved instance customization and QoL options

  
OStatus

   
OTP Releases

   
Refactor

   
Security

   
To Do

   
TwitterAPI

   
bug

   
complicated

   
confirmed

   
deps

   
discussion

   
documentation

   
easy

   
feature: to be merged in stable

   
fix: to be merged in stable

   
incident
Denotes a disruption to IT services and the associated issues require immediate attention

  
needs to be backported to maint

   
needs-changes

   
needs-info

   
needs-review

   
performance

   
regression

   
unclear
No labels
AP C2S
AdminAPI
Breaking API
Breaks existing config
Check if still valid
Data Protection
Do not merge
Document in the changelog
Doing
Feature Request / Enhancement
MastoAPI
NL2
NL4
OStatus
OTP Releases
Refactor
Security
To Do
TwitterAPI
bug
complicated
confirmed
deps
discussion
documentation
easy
feature: to be merged in stable
fix: to be merged in stable
incident
needs to be backported to maint
needs-changes
needs-info
needs-review
performance
regression
unclear
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
pleroma/pleroma!5885
No description provided.