Commit graph

18,218 commits

Author SHA1 Message Date
Lain Soykaf
e93f05a79f
fix(retention): keep threads whose post a local user quoted
A quote has a context of its own, so the quoted remote thread counted as
untouched and was evicted, breaking the local quote. On lain.com 45 of
134 quoted posts went this way. The objects_quote_url index keeps the
lookup at one probe per post.
2026-09-25 08:50:41 +04:00
Lain Soykaf
5e425ce9f5
fix(retention): see local reactions of the legacy EmojiReaction type
Older data stores emoji reactions as EmojiReaction. The per-thread check
only looked at EmojiReact, so a thread whose only local interaction was
such a reaction was evicted.
2026-09-23 15:43:02 +04:00
Lain Soykaf
5d1e331589
fix(retention): only delete objects that belong to an evicted thread
Eviction took the object ids of every activity in a thread. A Like or
Announce whose context differs from its object's then deleted a post in
another, possibly pinned thread. Found on a copy of a real instance:
593 posts from threads with local activity in the first 2M objects.
2026-09-23 13:45:02 +04:00
Lain Soykaf
5a4ada881f
test(retention): cover the recipients pin in prune_objects and watermark activity 2026-09-23 08:26:24 +04:00
Lain Soykaf
75b158c1d0
perf(retention): only clean up hashtags of evicted objects
Each run scanned the whole hashtags table. Collect the hashtag ids of
the objects about to be deleted and check just those.
2026-09-23 08:24:09 +04:00
Lain Soykaf
5067fd6d96
docs(retention): correct what the eviction transaction guarantees 2026-09-23 08:23:13 +04:00
Lain Soykaf
3cbcae2de3
fix(retention): keep threads younger than a day over the watermark
Over the watermark the walk went up to the present, so posts that had
just arrived could be evicted while people were reading them.
2026-09-23 08:23:05 +04:00
Lain Soykaf
1a0212bf9a
fix(retention): pin threads addressed to a local user
A mention or DM was only pinned by its notification, so clearing
notifications let retention evict a user's direct messages. Check the
activity recipients instead, which does not go away.
2026-09-23 08:22:14 +04:00
Lain Soykaf
89e684dc9c
docs(retention): describe the cursor walk and its limits 2026-09-02 12:32:09 +04:00
Lain Soykaf
6c8ed0a3b8
feat(retention): walk activities incrementally with a persisted cursor
Grouping the whole activities table by thread every run is unusable on
a large instance: on a 312 GB database it plans as a seq scan plus a
50M-row on-disk sort. Instead, walk activities in id order (flake ids
are time-ordered) from a persisted cursor up to the id that corresponds
to the deadline, and verify only the threads those activities belong to
through the (type, context) index. Measured on that instance: 147 ms
for a 50k-activity range scan, 349 ms warm / 14 s cold to verify ~700
threads.

The cursor is persisted per chunk so a killed run resumes. The
watermark walk keeps a cursor of its own so it cannot leave the
age-based walk stranded behind the deadline. Adds the retention_cursors
table.
2026-09-02 12:32:09 +04:00
Lain Soykaf
c1f8d75019
docs: document the bounded retention mode 2026-09-02 11:53:42 +04:00
Lain Soykaf
fa4fb00910
refactor(database): reuse retention query in prune_objects
prune_objects --keep-threads now shares the thread selection with
Pleroma.Retention, which additionally keeps threads that mention a
local user or that are referenced by a report. The hashtag cleanup
is shared as well.
2026-09-02 11:53:42 +04:00
Lain Soykaf
cd796f007d
feat(retention): run eviction from an Oban cron worker
Adds config :pleroma, :retention (disabled by default) and an hourly
crontab entry that evicts one bounded batch per run.
2026-09-02 11:53:42 +04:00
Lain Soykaf
1ebcd4f79d
feat(retention): add bounded eviction of old remote threads
Pleroma.Retention treats remote content as a cache: a thread is evicted
once nobody local has interacted with it and it has been quiet for
longer than remote_post_retention_days, or when the objects table is
over an optional row watermark. Local posts are never deleted.

A single grouping scan collects activity and object ids per thread so
the deletes only touch primary key and ap_id indexes.
2026-09-02 11:53:41 +04:00
lain
f53342775d Merge pull request 'Add rollback migrations for Rebased' (#7685) from gitlab-mr-iid-4308 into develop
Reviewed-on: https://git.pleroma.social/pleroma/pleroma/pulls/7685
2026-07-29 12:31:25 +00:00
Lain Soykaf
478b021fbe
fix(migrations): make Rebased rollback safe 2026-07-29 15:22:13 +04:00
nicole mikołajczyk
e1e853e8f7
Remove migration merged upstream
Signed-off-by: nicole mikołajczyk <git@mkljczk.pl>
2026-07-29 15:22:13 +04:00
mkljczk
2e59dcb9f6
style
Signed-off-by: mkljczk <git@mkljczk.pl>
2026-07-29 15:22:13 +04:00
mkljczk
e516c43411
Add docs for Rebased migration
Signed-off-by: mkljczk <git@mkljczk.pl>
2026-07-29 15:22:13 +04:00
mkljczk
253bbb80f5
Add rollback migrations for Rebased
Signed-off-by: mkljczk <git@mkljczk.pl>
2026-07-29 15:22:13 +04:00
lain
e143e645d3 Merge pull request 'OpenRC: Automatically run migrations before starting pleroma' (#7680) from gitlab-mr-iid-4303 into develop
Reviewed-on: https://git.pleroma.social/pleroma/pleroma/pulls/7680
2026-07-29 11:07:32 +00:00
Lain Soykaf
8943ab8140
fix(openrc): preserve migration execution context 2026-07-29 15:02:16 +04:00
lain
1dd6a345bf Merge pull request 'Respect activity restrictions in account statuses' (#7943) from mkljczk/pleroma:issue-2604-restrict-account-statuses into develop
Reviewed-on: https://git.pleroma.social/pleroma/pleroma/pulls/7943
2026-07-29 06:44:26 +00:00
Lain Soykaf
4a47a756be
fix(mastodon-api): optimize account status restrictions 2026-07-28 16:01:48 +04:00
nicole mikołajczyk
1bb5eca063 Merge pull request 'Fix typo in spec' (#7947) from mkljczk/pleroma:typo into develop
Reviewed-on: https://git.pleroma.social/pleroma/pleroma/pulls/7947
2026-07-28 10:41:05 +00:00
nicole mikołajczyk
5c85dadb29 Fix typo in spec
Signed-off-by: nicole mikołajczyk <git@mkljczk.pl>
2026-07-28 01:06:00 +02:00
lain
a45eb365ed Merge pull request 'Abort incomplete media proxy responses' (#7946) from lambadalambda/pleroma:fix/media-proxy-truncated-cache into develop
Reviewed-on: https://git.pleroma.social/pleroma/pleroma/pulls/7946
2026-07-27 15:08:06 +00:00
lain
a22062a224 Merge pull request 'OAuth App: Fix test, optimize code.' (#7692) from gitlab-mr-iid-4315 into develop
Reviewed-on: https://git.pleroma.social/pleroma/pleroma/pulls/7692
2026-07-27 14:42:53 +00:00
Lain Soykaf
bc999b3309
fix(media-proxy): abort truncated streams 2026-07-27 16:32:44 +04:00
lain
16581a2f6d Merge pull request 'Update Debian and Ubuntu installation requirements' (#7806) from gitlab-mr-iid-4429 into develop
Reviewed-on: https://git.pleroma.social/pleroma/pleroma/pulls/7806
2026-07-27 12:16:12 +00:00
Lain Soykaf
be5a3957b2
docs: explain libvips dependency 2026-07-27 16:12:21 +04:00
Lain Soykaf
3b71770cfa
docs: update Debian and Ubuntu requirements 2026-07-27 15:31:59 +04:00
Lain Soykaf
030c7e929c
style(oauth): use single alias directives 2026-07-27 12:14:58 +04:00
Lain Soykaf
d30d205f0b
fix(oauth): preserve dependent app cleanup 2026-07-27 11:57:16 +04:00
Lain Soykaf
b4c624911c
Merge branch 'develop' into gitlab-mr-iid-4315 2026-07-27 11:34:31 +04:00
lain
8ab0e046b1 Merge pull request 'Allow creating accounts without a password in the AdminAPI, update docs' (#7937) from mkljczk/pleroma:admin-api-passwordless-user-creation into develop
Reviewed-on: https://git.pleroma.social/pleroma/pleroma/pulls/7937
2026-07-27 07:17:14 +00:00
Phantasm
ea49ad121e Merge pull request 'Fix missing readme logo' (#7945) from readme-logo-fix into develop
Reviewed-on: https://git.pleroma.social/pleroma/pleroma/pulls/7945
2026-07-26 19:18:24 +00:00
Phantasm
6132c9e88a
Fix missing readme logo 2026-07-26 21:14:29 +02:00
Lain Soykaf
2b32e3cf1f
fix(mastodon-api): restrict unauthenticated account statuses 2026-07-26 18:12:49 +04:00
lain
fd2074a30d Merge pull request 'Mastodon-compatible webhooks' (#7844) from mkljczk/pleroma:webhooks into develop 2026-07-26 13:10:48 +00:00
Lain Soykaf
03e75d3b3e
fix(webhooks): merge develop and harden delivery 2026-07-26 17:00:14 +04:00
lain
b9ef1e2182 Merge pull request 'Restore preview cards tests and update their OpenAPI description' (#7444) from gitlab-mr-iid-4067 into develop
Reviewed-on: https://git.pleroma.social/pleroma/pleroma/pulls/7444
2026-07-26 10:32:26 +00:00
lain
e210e16a37 Merge pull request 'MastoAPI: /api/{v1,v2}/admin/accounts, /api/v1/admin/reports' (#7842) from mkljczk/pleroma:mastodon-admin-api into develop
Reviewed-on: https://git.pleroma.social/pleroma/pleroma/pulls/7842
2026-07-26 10:06:01 +00:00
Lain Soykaf
842e6bcaff
Merge remote-tracking branch 'origin/develop' into pr-7444-finish 2026-07-26 13:56:09 +04:00
Lain Soykaf
c591ca518c
fix(mastodon-api): finish preview card compatibility 2026-07-26 13:55:57 +04:00
Lain Soykaf
55c20c0fe8
test(mastodon-api): avoid relationship expiry races 2026-07-26 13:54:28 +04:00
lain
2ce6e5f49d Merge pull request 'Add ParadeDB search backend' (#7849) from paradedb-search-backend into develop
Reviewed-on: https://git.pleroma.social/pleroma/pleroma/pulls/7849
2026-07-26 09:28:06 +00:00
Lain Soykaf
458a15bf9f
fix(mastodon-api): harden admin moderation actions 2026-07-26 13:24:03 +04:00
Lain Soykaf
7a741736ea
Merge remote-tracking branch 'origin/develop' into pr-7444-finish 2026-07-26 13:20:50 +04:00
lain
3c916891f6 Merge branch 'develop' into paradedb-search-backend 2026-07-26 09:15:42 +00:00