Commit graph

220 commits

Author SHA1 Message Date
Lain Soykaf
4acd8c4e72
Log failed-signature retry rejections 2026-05-02 21:08:04 +04:00
Lain Soykaf
7756f491d5
Split failed-signature inbox retries
Route failed-signature ActivityPub inbox retries through a dedicated worker so legacy and malformed retry jobs fail closed before processing.
2026-05-01 08:43:42 +04:00
Phantasm
42683e79df
ReceiverWorker: Check that signature matches actor 2026-04-30 01:37:34 +02:00
Phantasm
799199f6b5 DigestEmailsWorker: Change Oban queue to "background"
The mailer queue has been long gone and that left Oban jobs always
stuck in the "available" state that would never execute.
2026-03-26 22:38:26 +00:00
Mark Felder
f06f0bedd3 Clean up ReachabilityWorker jobs and delete from Instances table when deleting all users and activities for an instance 2025-06-28 13:14:01 -07:00
Mark Felder
2267ace106 Ensure ReachabilityWorker jobs can be scheduled without needing awareness of the phase design 2025-06-27 18:12:18 -07:00
Mark Felder
a5e11ad110 Custom pruning is not actually needed because an old job cannot exist in the table due to our use of [replace: true] when retrying jobs or walking it through the different phases 2025-06-27 17:24:02 -07:00
Mark Felder
6e4b5edc25 Reduce pruning of history to anything older than 2 days 2025-06-27 17:13:34 -07:00
Mark Felder
77dca7c3e5 Refactor ReachabilityWorker to use a 5-phase reachability testing approach
It will check reachability for an instance deemed unreachable at the following intervals:

4 attempts, once a minute
4 attempts, once every 15 minutes
4 attempts, once every 60 minutes
4 attempts, once every 8 hours
4 attempts, once every 24 hours

This should be effective and respectful of the resources of instances on the fediverse.

We have the Oban Pruner plugin enabled to keep the Oban Jobs table from growing indefinitely. It prunes every 15 minutes, but this will interfere with our ability to enforce uniqueness on the ReachabilityWorker jobs for a time period longer than 15 minutes. The solution is to exclude the ReachabilityWorker from the pruning operation and instead schedule a custom job that will prune the table for us once a day. The ReachabilityPruner cron task will clean up the history of the ReachabilityWorker jobs older than 6 days.
2025-06-27 17:00:02 -07:00
Mark Felder
59bfa83c9c Remove daily reachability scheduling for unreachable instances 2025-06-27 16:04:08 -07:00
Mark Felder
e58ecd3234 Merge remote-tracking branch 'origin/develop' into improved-reachability 2025-06-27 15:59:46 -07:00
Mark Felder
56aab905e8 Queue individual jobs for each user that needs to be deleted when deleting an instance. 2025-06-27 11:13:15 -07:00
Mark Felder
a46a48fb3f PublisherWorker: change max_attempts to 13 which extends the last delivery attempt to ~4.3 days 2025-06-07 15:13:45 -07:00
Mark Felder
8383584d69 Reapply "Nodeinfo is not universally implemented"
This reverts commit 0fe03fc4ee.
2025-06-07 14:57:34 -07:00
Mark Felder
0fe03fc4ee Revert "Nodeinfo is not universally implemented"
This reverts commit b87ec49972.
2025-06-06 13:44:24 -07:00
Mark Felder
0f667761a9 The ap_id is a URL, so we can just pass that to set_reachable/1
Also only bother attempting to mark reachable if it was known to be unreachable
2025-06-06 13:01:32 -07:00
Mark Felder
b87ec49972 Nodeinfo is not universally implemented 2025-06-06 12:55:21 -07:00
Mark Felder
3d422ef325 Reachability refactor
The result of Oban jobs determine the reachability status.

Publisher jobs will cancel themselves at execution time if the target server is now unreachable.

Receiving activities does not immediately mark a server as reachable, but creates a ReachabilityWorker job to validate.

A Cron will execute daily to test all unreachable servers.
2025-06-06 12:32:09 -07:00
Ekaterina Vaartis
51a0cee405 Add expiring blocks
- `/api/v1/accounts/:id/block` now has a "duration" parameter
- `/api/v1/blocks` returns "block_expires_at" to indicate when the block
will expire
- MuteExpireWorker also processes block expiration
- Remove unused OpenAPI parameters from mute endpoint
- Add pleroma:block_expiration to nodeinfo features
2025-04-15 12:55:09 +03:00
Mark Felder
03a6e33b81 Skip the final refresh job if the activity is local 2024-10-09 16:25:58 -04:00
Mark Felder
b854e3836f Remove pattern that can never match 2024-10-03 10:30:32 -04:00
Mark Felder
ba2ae5e40b Check if a refresh is permitted by comparing timestamps before attempting to insert an Oban job
It's better to avoid inserting an Oban job that will just be rejected if it's not expensive to check.
2024-10-03 10:14:02 -04:00
Mark Felder
a2e7db43aa Rename assignment for consistency 2024-09-30 14:23:04 -04:00
Mark Felder
47ce3a4a96 Schedule a final poll refresh before streaming out the notifications 2024-09-30 14:17:35 -04:00
Mark Felder
4b3f604f95 Skip refetching poll results if the object's updated_at is newer than the poll closed timestamp 2024-09-30 14:02:41 -04:00
Mark Felder
c077a14ce1 Add Oban job to handle poll refreshing and stream out the update 2024-09-30 13:54:56 -04:00
Mark Felder
a887188890 Oban: more unique job constraints 2024-09-06 09:42:53 -04:00
Mark Felder
a1951f3af7 Add Cron worker to clean up orphaned apps hourly 2024-09-04 10:59:58 -04:00
Mark Felder
1821ef4f15 Move user active check into Federator.perform/1 2024-08-28 18:35:09 -04:00
Mark Felder
8a3efa7152 More error annotations 2024-08-28 18:02:35 -04:00
Mark Felder
c5ca806aa0 Add back one of the duplicate checks to fix a test, document where it comes from 2024-08-28 17:57:34 -04:00
Mark Felder
380a6a6df3 :validate_object is not a real error returned from anywhere 2024-08-28 17:45:31 -04:00
Mark Felder
2346807ac9 Annotate error cases 2024-08-28 17:44:33 -04:00
Mark Felder
6ae629cfe0 Cancel ReceiverWorker jobs if the user account has been disabled / deactivated 2024-08-28 17:24:59 -04:00
Mark Felder
fc450fdefc ReceiverWorker: cancel job if user fetch is forbidden
An instance block with authenticated fetch being required can cause this as we couldn't get the user to find their public key to verify the signature. Commonly observed if someone boosts/Announces a post from an instance that blocked you.
2024-08-28 15:45:16 -04:00
Mark Felder
a9aa810d3d Change imports to generate an Oban job per each task 2024-08-22 13:24:07 -04:00
Mark Felder
089fa4d146 Improve Remote Object Fetcher error handling, Oban 2024-08-17 20:42:33 -04:00
Mark Felder
52e9bec156 Remove WorkerHelper 2024-07-30 11:34:48 -04:00
feld
c9042763b0 Merge branch 'oban/backup' into 'develop'
Backups Refactoring

See merge request pleroma/pleroma!4158
2024-07-29 16:07:35 +00:00
Mark Felder
775f45cfe2 Merge remote-tracking branch 'origin/develop' into oban/backup 2024-07-25 14:43:40 -04:00
Mark Felder
a964368e31 Add test to fetch and validate an activity that originally failed signature 2024-07-25 14:34:55 -04:00
Mark Felder
1b9c887dbb Extract validate_signature/2 from the HTTPSignaturePlug
This logic only exists in the Plug, so attempting to validate the signature by calling the library function HTTPSignature.validate_conn/2 directly will never work because we do not attempt to construct the (request-target) and @request-target headers with both the commonly misinterpreted and correct implementation of this field. Therefore all attempts to validate a signature from an Oban Job will fail.
2024-07-25 12:54:27 -04:00
Mark Felder
1a482a73c3 Fix Optimistic Inbox for failed signatures
When signatures fail on incoming activities we put the job into Oban to be processed later instead of doing the user fetching and validation inline which is expensive and increases latency on the incoming POST request. Unfortunately we did not retain the :method, :request_path, and :query_string parameters from the conn so the signature validation and Oban Job would always fail.

This was most obvious when Mastodon sends Deletes for users your server has never seen before.
2024-07-25 11:46:30 -04:00
Mark Felder
5a62868106 Consider errors during HTTP GET and HEAD to be unrecoverable and insert a negative cache entry
This is for a normal HTTP error response or timeout while receiving the data. A hard error from a process crash, DNS lookup failure, etc should produce a different response than {:ok, %Tesla.Env{}} and the request/job will be retryable.
2024-07-24 15:43:49 -04:00
Mark Felder
731f7b87d2 Pad RichMediaWorker timeout to be 2s longer than the Rich Media HTTP timeout 2024-07-24 15:42:50 -04:00
Mark Felder
187897874b Make backup timeout configurable 2024-07-23 10:30:57 -04:00
Mark Felder
e5cbbaf3f0 Extend the backup job time limit to 30 minutes 2024-07-23 10:30:57 -04:00
Mark Felder
ece063586b Limit backup jobs to 5 minutes 2024-07-23 10:30:57 -04:00
Mark Felder
8f285a787f Refactor backups to be fully controlled by Oban 2024-07-23 10:30:40 -04:00
Mark Felder
d27ad36ce4 Fix order of args for remove_mute/2 2024-07-22 18:25:37 -04:00