After a redirect the 429/503 came from the final host, but the backoff
was recorded for the requested one. X-RateLimit-Reset is also accepted
as a Unix timestamp or as seconds.
Ported from Akkoma af7c3fab9; the rescue is in the Ecto type load so
admin input is still validated.
Co-authored-by: FloatingGhost <hannah@coffee-and-dreams.uk>
Gun resolves host names over IPv4 only unless :inet6 is in tcp_opts.
When the A lookup finds nothing, retry once with :inet6. Gun now runs
under its supervisor, so a failed connect no longer kills the pool
worker through the link; the worker still links after a successful open.
A Move usually follows right after the alias was added, and the cached
remote target may not have it yet. Refetch the target when the alias is
missing instead of rejecting the Move. Same issue as Akkoma f18e2ba42.
Bridgy Fed delivers Follow and Accept to the personal inbox addressed
only to Public. Any server can already address a local user directly,
so the check gave no protection. Ported from Akkoma 2b885288f.
Co-authored-by: Oneric <oneric@oneric.stub>
With 5 attempts and Oban's short retry delays, an incoming activity from
an unknown actor on a backed-off host was discarded without a request.
ReceiverWorker and SignatureRetryWorker now snooze for up to 2 hours.
Also lowercase backoff host keys and log backed-off user fetches at debug.
Object fetches and WebFinger stop sending requests to a host until
Retry-After or X-RateLimit-Reset (default 5 minutes, max 1 hour).
Ported from Akkoma's Pleroma.HTTP.Backoff.
Co-authored-by: FloatingGhost <hannah@coffee-and-dreams.uk>
Co-authored-by: Oneric <oneric@oneric.stub>
Escape the domain so `.` only matches a dot, and use an optional
group for the `*.` prefix, which matches the same hosts without
exponential backtracking. Bad entries now match nothing instead of
raising on every check.
Without them replace_params: false was ignored when strict is off, so
conn.params got atom keys and RateLimiter params: ["id"] buckets
collapsed to one per user.
the :do_not_federate checks were omitted from the undo pipeline,
which could lead to them federating.
this commit enforces :outgoing_blocks on undos as well - and
any existing value of :do_not_federate is preserved, if one exists
Akkoma issue 957.
[Pleroma: block/unblock test calls use the Pleroma (blocked, blocker) argument order.]
(cherry picked from commit eca7ed572b8a10a0235fb1a381eac4803614dee4)
Status search fetched any http(s) query from its remote server, for
anonymous users, on every result page and with resolve=false, so anyone
could make the server fetch URLs. As in Mastodon and in Pleroma's user
search, fetch only with resolve=true, on the first page, and for
anonymous users only if [:instance, :limit_to_local_content] is false.
Based on Akkoma ca9e33ed2 and 17efec060.
Co-authored-by: Oneric <oneric@oneric.stub>
The emoji reaction create and delete endpoints had no rate limiter.
Apply :statuses_actions like the other status actions, plus a per-status
:status_id_action bucket ("status_id_action:emoji_react") in the same way
favourite and reblog do.
POST /users/:nickname/outbox had no rate limiter, so a client could send
unlimited activities through the C2S API. Apply :statuses_actions to
update_outbox. RateLimiter keys buckets by limiter name and user, so the
outbox shares one counter with the Mastodon API status actions and mixing
both APIs does not double the limit.
StatusController keeps string param keys (replace_params: false, since
b8db67daf), so params: [:id] never matched and status_id_action became
one bucket per user across all statuses (3 favourites/reblogs a minute).
The limit was a connection parameter, so it applied to every GIN index
scan, not only full-text search: recipient lookups and Oban job queries
could return incomplete, random results. Set it with SET LOCAL around
the search query only, configurable under Pleroma.Search.DatabaseSearch.
The default stays 500 for search: measured on a 74M-object database,
higher limits or none make searches for common words exceed the query
timeout. Remote fetches for URL queries run outside the transaction.
Based on Akkoma 811f0a468.
Co-authored-by: Oneric <oneric@oneric.stub>
The peers query read the nickname of every remote user every minute
(2.7 s and 1.2M buffer hits on a large instance). The instances table
now has a row for every host we know users from, so read it instead,
and recompute the stats every 5 minutes.
Based on Akkoma 138b1aea2 and 7b76fdeed.
Co-authored-by: Oneric <oneric@oneric.stub>
Co-authored-by: FloatingGhost <hannah@coffee-and-dreams.uk>
The instances table only got rows from fetches, reachability changes
and favicon lookups, so most hosts we know users from had none. Insert
a row when a remote user is created, keeping existing rows.
Mastodon 4.4+ clients read quotes from a top-level quote object with
state and quoted_status; Pleroma only had pleroma.quote, so they did
not show quote posts. Quoted statuses get a shallow quote with only
quoted_status_id, as quotes are rendered one level deep.
Inspired by Akkoma 5b7209980, which can not use the shallow form because
Akkoma already had a top-level quote status.
Containment.get_actor raised on remote shapes it did not know, such as
an attributedTo object without an id. This crashed object fetches and
poll refetches. Return nil, so containment rejects the object.
A fetch from a display URL (e.g. https://host/@user) returns the
actor with its canonical id. make_user_from_ap_id looked the user up
by the fetch URL, did not find it and tried to insert a duplicate.
Look the user up by the fetched id.
Based on Akkoma 48b3a3579.
refetch_object (used for poll updates) accepted whatever the remote
served. A hostile server could swap in another object or move the post
to another author.
The refetch now goes to the stored id without redirects, and the
returned id and actor must equal the stored ones.
Based on Akkoma 9061d148b and fee57eb37.
The fetcher accepted any document whose id was on the same host as the
fetch URL. Anyone who can place a JSON file on a host, for example
through an upload, could pass it off as another object or actor from
that host.
The id of a fetched document must now be the URL it came from (after
redirects), ignoring a trailing slash, the fragment, the host case and
differences that our URL encoding removes.
If it differs, as for Mastodon's /@user/123 display URLs, we refetch
once from that id, which must be on the same host. The refetch must
not redirect and must match exactly.
Test mocks that served a document under a different URL now serve it
from its own id.
Based on Akkoma 8684964c5 and 940792f8b.
