* Whenever a remote keypair changes, unfollow them and re-subscribe to them
In Mastodon (it could be different for other OStatus or AP-enabled software),
a keypair change is indicative of whole user (or instance) data loss. In this
situation, the “new” user might be different, and almost certainly has an empty
followers list. In this case, Mastodon instances will disagree on follower
lists, leading to unreliable delivery and “shadow followers”, that is users
believed by a remote instance to be followers, without the affected user
knowing.
Drawbacks of this change are:
1. If an user legitimately changes public key for some reason without losing
data (not possible in Mastodon at the moment), they will have their remote
followers unsubscribed/re-subscribed needlessly.
2. Depending of the number of remote followers, this may generate quite some
traffic.
3. If the user change is an attempt at usurpation, the remote followers will
unknowingly follow the usurper. Note that this is *not* a change of
behavior, Mastodon already behaves like that, although delivery might be
unreliable, and the usurper would not have known the former user's
followers.
* Rename ResubscribeWorker to RefollowWorker
* Process followers in batches
When a new user confirms their e-mail, bootstrap their home timeline
by automatically following a set of accounts. By default, all local
admin accounts (that are unlocked). Can be customized by new admin
setting (comma-separated usernames, local and unlocked only)
- Fix assumption that `url` is always a string. Handle it if it's an
array of strings, array of objects, object, or string, both for
accounts and for objects
- `sharedInbox` is actually supposed to be under `endpoints`, handle
both cases and adjust the serializer
* Make "unfollow" undo pending outgoing follow request too
* Add cancel button to web UI when awaiting follow request approval
* Make the hourglass button do the cancelling
Using _: property names is discouraged, as in the future,
canonicalization may throw an error when encountering that instead
of discarding it silently like it does now.
We are defining some ActivityStreams properties which we expect
to land in ActivityStreams eventually, to ensure that future versions
of Mastodon will remain compatible with this even once that happens.
Those would be `locked`, `sensitive` and `Hashtag`
We are defining a custom context inline for some properties which we
do not expect to land in any other context. `atomUri`, `inReplyToAtomUri`
and `conversation` are part of the custom defined OStatus context.
* Make PreviewCard records reuseable between statuses
**Warning!** Migration truncates preview_cards tablec
* Allow a wider thumbnail for link preview, display it in horizontal layout (#4648)
* Delete preview cards files before truncating
* Rename old table instead of truncating it
* Add mastodon:maintenance:remove_deprecated_preview_cards
* Ignore deprecated_preview_cards in schema definition
* Fix null behaviour
* Guarantee Subscription service first account has proper URL details
Subscription Service potentially could break if the first user suspended
themselves, creating a situation where the urls that populate throughout
subscription service's PuSH request would cause the remote API to throw 503 errors.
Guaranteeing that the first account picked is not suspended prevents this problem.
* Fix style issue
Requires moving Atom rendering from DistributionWorker (where
`stream_entry.status` is already nil) to inline (where
`stream_entry.status.destroyed?` is true) and distributing that.
Unfortunately, such XML renderings can no longer be easily chained
together into one payload of n items.
* Add handling of Linked Data Signatures in payloads
* Add a way to sign JSON, fix canonicalization of signature options
* Fix signatureValue encoding, send out signed JSON when distributing
* Add missing security context
* Process Create / Announce activity in FetchRemoteStatusService
* Use activity URL in ActivityPub for reblogs
* Redirect to the original status on StatusesController#show
* Ignore empty response in ActivityPub::FetchRemoteStatusService
This fixes `NoMethodError: undefined method `[]' for nil:NilClass` error.
* Check json.nil? in JsonLdHelper#supported_context?
* Fallback to OStatus in FetchAtomService
* Skip activity+json link if that activity is Person without inbox
* If unsupported activity was detected and all other URLs failed, retry with ActivityPub-less Accept header
* Allow mention to OStatus account in ActivityPub
* Don't update profile with inbox-less Person object
- Tries to avoid performing HTTP request if the keyId is an actor URI
- Likewise if the URI is a fragment URI on top of actor URI
- Resolves public key, returns owner if the owner links back to the key
* ActivityPub migration procedure
Once one account is detected as going from OStatus to ActivityPub,
invalidate WebFinger cache for other accounts from the same domain
* Unsubscribe from PuSH updates once we receive an ActivityPub payload
* Re-subscribe to PuSH unless already unsubscribed, regardless of protocol
* Add ActivityPub inbox
* Handle ActivityPub deletes
* Handle ActivityPub creates
* Handle ActivityPub announces
* Stubs for handling all activities that need to be handled
* Add ActivityPub actor resolving
* Handle conversation URI passing in ActivityPub
* Handle content language in ActivityPub
* Send accept header when fetching actor, handle JSON parse errors
* Test for ActivityPub::FetchRemoteAccountService
* Handle public key and icon/image when embedded/as array/as resolvable URI
* Implement ActivityPub::FetchRemoteStatusService
* Add stubs for more interactions
* Undo activities implemented
* Handle out of order activities
* Hook up ActivityPub to ResolveRemoteAccountService, handle
Update Account activities
* Add fragment IDs to all transient activity serializers
* Add tests and fixes
* Add stubs for missing tests
* Add more tests
* Add more tests
* Allow domain blocks to reject media without silencing or suspending
* Fix typo
* Hide 'Reject media' button when superfluous, instead of disabling it
* Properly hide 'reject media' checkbox on page load if needed
This may happen when resubmitting the domain block form after an error.
* Don't ask whether undoing a media-only block should be retroactive
* Rename :media_only block to :noop
* Display :noop block as None in frontend, allow blocks that do nothing
* Remove 'coding' line auto-added by emacs
* Do not raise unretryable exceptions in ResolveRemoteAccountService
* Removed fatal exceptions from ResolveRemoteAccountService
Exceptions that cannot be retried should not be raised. New exception
class for those that can be retried (Mastodon::UnexpectedResponseError)
* Refactor ResolveRemoteAccountService
* Remove trailing whitespace
* Use redis locks around critical ResolveRemoteAccountService code
* Add test for race condition of lock
* Wrap methods of ProcessFeedService::ProcessEntry in classes
This is a change same with 425acecfdb, except
that it has the following changes:
* Revert irrelevant change in find_or_create_conversation
* Fix error handling for RemoteActivity
* Introduce Ostatus name space
* Add dependency on idn-ruby to speed up URI normalization
* Use normalized_host instead of normalize.host when applicable
When we are only interested in the normalized host, calling normalized_host
avoids normalizing the other components of the URI as well as creating a
new object
* Add Request class with HTTP signature generator
Spec: https://tools.ietf.org/html/draft-cavage-http-signatures-06
* Add HTTP signature verification concern
* Add test for SignatureVerification concern
* Add basic test for Request class
* Make PuSH subscribe/unsubscribe requests use new Request class
Accidentally fix lease_seconds not being set and sent properly, and
change the new minimum subscription duration to 1 day
* Make all PuSH workers use new Request class
* Make Salmon sender use new Request class
* Make FetchLinkService use new Request class
* Make FetchAtomService use the new Request class
* Make Remotable use the new Request class
* Make ResolveRemoteAccountService use the new Request class
* Add more tests
* Allow +-30 seconds window for signed request to remain valid
* Disable time window validation for signed requests, restore 7 days
as PuSH subscription duration (which was previous default due to a bug)
- Use unicode when selecting emoji through picker
- Convert shortcodes to unicode when storing text input server-side
- Do not convert shortcodes in JS anymore
* Specs for language detection
* Use CharlockHolmes instead of NKF
* Correct mistakes
* Correct style
* Set hint_enc instead of falling back and strip_tags
* Improve specs
* Add dependencies
Ensure the only allowed author of top-level entries in feed is the person
the feed belongs to (a verified user). Ensure delete events only apply
if the deleted item belonged to that user.
The classes using Status.as_home_timeline, namely Feed and
PrecomputeFeedService are expected to filter direct statuses as
FanOutWriteService does, but their filtering were incomplete or missing.
This commit solves the problem by filtering direct statuses in
as_home_timeline as the other similar methods such as as_public_timeline
does.
* Make Pubsubhubbub::DistributionWorker handle both single stream entry
arguments, as well as arrays of stream entries
* Add BatchedRemoveStatusService, make SuspendAccountService use it
* Improve method names
* Add test
* Add more tests
* Use PuSH payloads of 100 to have a clear mapping of
1000 input statuses -> 10 PuSH payloads
It was nice while it lasted
* Add form for account deletion
* If avatar or header are gone from source, remove them
* Add option to have SuspendAccountService remove user record, add tests
* Exclude suspended accounts from search
* Fix#2619 - When redis feed is empty, fall back to database
* Use redis value to return feed from database only while RegenerationWorker
hasn't finished running
* Fix specs
* Replace usage of reject!
TagManager.local_url? was sometimes called with an URI with a nil host,
leading to a crash in TagManager.local_url?. This fixes moves the
already-existing uri.host.blank? check in front to avoid this case.
Steps to reproduce the original issue:
1. Have two remote accounts, A that you don't follow, and B that you follow.
2. Have A post a toot and reply to it.
3. Boost A's reply from remote account B.
This used to cause the local instance to get A's reply but fail to link it to
the original post.
* Try fixing ThreadResolveWorker calls
From my understanding of ActiveRecord, a transaction is commited as soon as
the exit of the outmost ActiveRecord.transaction block. However, inner
transaction blocks will exit without the transaction being commited.
In this case, ThreadResolveWorker were fired *within* a transaction block,
so moving the call out of it should do the trick. However, this is somewhat
fragile, as this whole codepath could be called within yet another transaction.
* Set status thread within the transaction block if it is immediately available from database
* Remove trailing whitespace in i18n mailers
* Use query methods instead of #present? on AR attributes
* Delegate Status#account_domain method
* Delegate Mention #account_username and #account_acct methods
* Fix nil input not handled well in AuthorExtractor concern
* Fix hard error in ProcessFeedService when replied-to status has been deleted
* Fix nil errors in ProcessInteractionService when favourited status
cannot be found
* Add buttons to block and unblock domain
* Relationship API now returns "domain_blocking" status for accounts,
rename "block entire domain" to "hide entire domain", fix unblocking domain,
do not block notifications from domain-blocked-but-followed people, do
not send Salmons to domain blocked users
* Add test
* Personal domain blocks shouldn't affect Salmon after all, since in this
direction of communication the control is very thin when it comes to
public stuff. Best stay consistent and not affect federation in this way
* Ignore followers and follow request from domain blocked folks,
ensure account domain blocks are not created for empty domain,
and avoid duplicates in validation
* Purge followers when blocking domain (without soft-blocks, since they
are useless here)
* Add tests, fix local timeline being empty when having any domain blocks
* Add <ostatus:conversation /> tag to Atom input/output
Only uses ref attribute (not href) because href would be
the alternate link that's always included also.
Creates new conversation for every non-reply status. Carries
over conversation for every reply. Keeps remote URIs verbatim,
generates local URIs on the fly like the rest of them.
* Conversation muting - prevents notifications that reference a conversation
(including replies, favourites, reblogs) from being created. API endpoints
/api/v1/statuses/:id/mute and /api/v1/statuses/:id/unmute
Currently no way to tell when a status/conversation is muted, so the web UI
only has a "disable notifications" button, doesn't work as a toggle
* Display "Dismiss notifications" on all statuses in notifications column, not just own
* Add "muted" as a boolean attribute on statuses JSON
For now always false on contained reblogs, since it's only relevant for
statuses returned from the notifications endpoint, which are not nested
Remove "Disable notifications" from detailed status view, since it's
only relevant in the notifications column
* Up max class length
* Remove pending test for conversation mute
* Add tests, clean up
* Rename to "mute conversation" and "unmute conversation"
* Raise validation error when trying to mute/unmute status without conversation
* Adding account domain blocks that filter notifications and public timelines
* Add tests for domain blocks in notifications, public timelines
Filter reblogs of blocked domains from home
* Add API for listing and creating account domain blocks
* API for creating/deleting domain blocks, tests for Status#ancestors
and Status#descendants, filter domain blocks from them
* Filter domains in streaming API
* Update account_domain_block_spec.rb
* Add <ostatus:conversation /> tag to Atom input/output
Only uses ref attribute (not href) because href would be
the alternate link that's always included also.
Creates new conversation for every non-reply status. Carries
over conversation for every reply. Keeps remote URIs verbatim,
generates local URIs on the fly like the rest of them.
* Conversation muting - prevents notifications that reference a conversation
(including replies, favourites, reblogs) from being created. API endpoints
/api/v1/statuses/:id/mute and /api/v1/statuses/:id/unmute
Currently no way to tell when a status/conversation is muted, so the web UI
only has a "disable notifications" button, doesn't work as a toggle
* Display "Dismiss notifications" on all statuses in notifications column, not just own
* Add "muted" as a boolean attribute on statuses JSON
For now always false on contained reblogs, since it's only relevant for
statuses returned from the notifications endpoint, which are not nested
Remove "Disable notifications" from detailed status view, since it's
only relevant in the notifications column
* Up max class length
* Remove pending test for conversation mute
* Add tests, clean up
* Rename to "mute conversation" and "unmute conversation"
* Raise validation error when trying to mute/unmute status without conversation
* Add <ostatus:conversation /> tag to Atom input/output
Only uses ref attribute (not href) because href would be
the alternate link that's always included also.
Creates new conversation for every non-reply status. Carries
over conversation for every reply. Keeps remote URIs verbatim,
generates local URIs on the fly like the rest of them.
* Fix conversation migration
* More spec coverage for status before_create
* Prevent n+1 query when generating Atom with the new conversations
* Improve code style
* Remove redundant local variable
* Fix regressions from #2683
Properly format spoiler text HTML, while keeping old logic for blankness intact
Process hashtags and mentions in spoiler text
Format spoiler text for Atom
Change "show more" toggle into a button instead of anchor
Fix style regression on dropdowns for detailed statuses
* Fix lint issue
* Convert spoiler text to plaintext in desktop notifications
* services: scan spoiler_text for hashtags (#699)
* views: link hashtags from spoiler_texts
This covers linking hashtags from within the spoiler
text on the server-generated pages.
* services: fix string concat going into hashtag RE
Cleaner Ruby syntax, may handle immutable strings better
* Fix#2473 - Use sidekiq scheduler to refresh PuSH subscriptions instead of cron
Fix an issue where / in domain would raise exception in TagManager#normalize_domain
PuSH subscriptions refresh done in a round-robin way to avoid hammering a single
server's hub in sequence. Correct handling of failures/retries through Sidekiq (see
also #2613). Optimize Account#with_followers scope. Also, since subscriptions
are now delegated to Sidekiq jobs, an uncaught exception will not stop the entire
refreshing operation halfway through
Fix#2702 - Correct user agent header on outgoing http requests
* Add test for SubscribeService
* Extract #expiring_accounts into method
* Make mastodon:push:refresh no-op
* Queues are now defined in sidekiq.yml
* Queues are now in sidekiq.yml
* Likely fix#2458, fix#2031 - handle out-of-order deletes for statuses
If a delete arrives before the original status, cache that information
for 6h, and if the original status arrives in that window, ignore it
* Add test case
Fix#2196 - Respond with 201 when Salmon accepted, 400 when unverified
Fix#2629 - Correctly handle confirm_domain? for local accounts
Unify rules for extracting author acct from XML, prefer <email>, fall back
to <name> + <uri> (see also #2017, #2172)
Follow-up to #2599. When a domain block with `reject_media` is
added or `rake mastodon:media:remove_remote` is invoked, mastodon
deletes the locally cached attachments and avatars but does not
reflect that change in the database, causing the `file` fields to
still have values. This change persists the deletion in the
database and sets the attachment type to unknown.
This also introduces a one-off rake task that sets all attachments
without a local file to the "unknown" type. The upgrade notes for
the next release should contain a post-upgrade step with
`rake mastodon:media:set_unknown`.
* Instead of parsing shared status contents verbatim, make roundtrip
to purported original URL. Confirm that the "original" URL is from the
same domain as the author it claims to be from.
* Fix obvious typo, add comment
* Use URI look-up first
* Add test, update Goldfinger dependency to make less useless HTTP requests per Webfinger lookup
* OEmbed support for PreviewCard
* Improve ProviderDiscovery code failure treatment
* Do not crawl links if there is a content warning, since those
don't display a link card anyway
* Reset db schema
* Fresh migrate
* Fix rubocop style issues
Fix#1681 - return existing access token when applicable instead of creating new
* Fix test
* Extract http client to helper
* Improve oembed controller
* Add coverage for domain block service with silence
* Get rid of warning about find_each and order
* Move domain_block to attr_reader
* Move optional clear_media into silence_accounts method
* Use blocked_domain method to reduce passed vars
* Extract blocked_domain_accounts method to find accounts on the domain
* Extract media_from_blocked_domain method to find relevant attachments
* Separate destruction of account images and account attachments
duplicates. Web UI regenerates UUID for that header every time the compose
form is changed or successfully submitted
Also, fix Farsi i18n overwriting the English one
* Fix#2119 - Whenever about to send a HTTP request, normalize the URI
* Add test for IDN request in FetchLinkCardService
* Perform IDN normalization on domains before they are stored in the DB
* Fix#1057 (close#1819) - Move HTML-formatted bio from <poco:note /> to <summary type="html" />
* Ensure <poco:note /> is plaintext for remote accounts, also, by stripping out HTML
* Extract detect_language to separate class
* Use default locale, not just en
* Add spec to confirm that whatlanguage cant identify empty string
* Allow account locale to override default in language detector
* PostStatusService supplies an account to detect language
