[:http, :adapter, :ssl_options] is Hackney's key, and AdminFE sets it.
Gun reads tls_opts, so with Gun as the default these settings were lost
without a warning. Use them as tls_opts unless tls_opts is set.
Hackney 4 reads every response completely before Pleroma can check its
size, so max_body no longer limits downloads or memory there; Gun stops
at the limit. Document the Hackney limitation for those who switch back.
FastSanitize checks the scheme with URI.parse/1, which finds none in
java<TAB>script:alert(1). Browsers remove tabs and newlines and leading
control characters first and run it as a javascript: link. The scrubbers
now use Pleroma.HTML.URIAttributes, which normalizes the URL the same
way before the scheme check.
Cutting the URL at the first quote dropped the rest of valid links such
as ?q="cat"&page=2. Percent-encoding ", < and > keeps the URL
complete and still keeps it inside the attribute.
The email autolink clause also matched a link whose only child is an
element, raised on "mailto:" <> element, and the rescue turned the
whole post into escaped source text.
Hackney 4 negotiates HTTP/2 and shares one connection per host. The
media proxy closes a response with :hackney.close/1, which stops the
whole connection, so parallel downloads from one host broke each other.
ConfigDB turns a numeric proxy host into an IP tuple. Hackney 1 took it,
Hackney 4 ignores such a proxy and connects directly, which skips a
SOCKS proxy (Tor) without any error.
- An admin-set cacerts no longer gets the CAStore cacertfile added;
Hackney drops cacerts when both are set.
- Set proxy_ssl_options too: with an HTTPS proxy, Hackney 4 uses them
for the proxy connection and falls back to certifi.
- Hackney 4 ignores the top-level insecure option and uses certifi.
Turn it into an explicit verify_none.
The Qdrant and OpenAI clients do not go through Pleroma.HTTP, so on
Hackney they trusted certifi. Both send API keys. Give them the
configured adapter with the same TLS options.
Hackney 4 merges given ssl_options into its defaults and keeps its own
partial_chain, which trusts any certificate in the peer chain that is in
certifi. A peer could send the emSign ECC Root CA - G3 root in its chain
and pass. Set partial_chain so certifi is never a trust anchor.
The TLS version override replaced all Hackney TLS defaults. On OTP 26+
this fails with cacerts: :undefined; on older OTP it skipped certificate
checks. Closes#7962 on the public tracker.
Hackney trusts its own certifi bundle, which still has the roots of the
compromised eMudhra intermediate CAs. Pass explicit ssl_options with the
CAStore bundle. Hackney drops all its TLS defaults when ssl_options are
given, so keep peer and hostname checks in the options.
Mastodon, Misskey and InlineQuotePolicy put "RE: <url>" in a .quote-inline
element at the start of a quote post. Its link was taken for the link card,
which then showed the quoted post a second time.
A quote has a context of its own, so the quoted remote thread counted as
untouched and was evicted, breaking the local quote. On lain.com 45 of
134 quoted posts went this way. The objects_quote_url index keeps the
lookup at one probe per post.
Older data stores emoji reactions as EmojiReaction. The per-thread check
only looked at EmojiReact, so a thread whose only local interaction was
such a reaction was evicted.
Eviction took the object ids of every activity in a thread. A Like or
Announce whose context differs from its object's then deleted a post in
another, possibly pinned thread. Found on a copy of a real instance:
593 posts from threads with local activity in the first 2M objects.
A mention or DM was only pinned by its notification, so clearing
notifications let retention evict a user's direct messages. Check the
activity recipients instead, which does not go away.
Wrap the followed ap_ids subselect in keep_following_or_not_domain_blocked
in ARRAY(...) so it becomes an InitPlan. As a plain subselect the planner
charged its startup per rescan of the actor lookup, which could make
Notification.for_user pick a plan that scans every active user per row.
Grouping the whole activities table by thread every run is unusable on
a large instance: on a 312 GB database it plans as a seq scan plus a
50M-row on-disk sort. Instead, walk activities in id order (flake ids
are time-ordered) from a persisted cursor up to the id that corresponds
to the deadline, and verify only the threads those activities belong to
through the (type, context) index. Measured on that instance: 147 ms
for a 50k-activity range scan, 349 ms warm / 14 s cold to verify ~700
threads.
The cursor is persisted per chunk so a killed run resumes. The
watermark walk keeps a cursor of its own so it cannot leave the
age-based walk stranded behind the deadline. Adds the retention_cursors
table.
prune_objects --keep-threads now shares the thread selection with
Pleroma.Retention, which additionally keeps threads that mention a
local user or that are referenced by a report. The hashtag cleanup
is shared as well.
Pleroma.Retention treats remote content as a cache: a thread is evicted
once nobody local has interacted with it and it has been quiet for
longer than remote_post_retention_days, or when the objects table is
over an optional row watermark. Local posts are never deleted.
A single grouping scan collects activity and object ids per thread so
the deletes only touch primary key and ap_id indexes.
A partial revert, keeping the updated @inline_content_types list and
cosmetic changes.
This reverts commit 4fe567064a, reversing
changes made to 9b8aa2e2ee.
