Commit 0b2119d4 authored by Haelwenn's avatar Haelwenn

Merge branch 'release/2.4.1' into 'stable'

Release: 2.4.1

See merge request !3501
parents dc63aaf8 7372609c
Pipeline #37347 passed with stages
in 12 minutes and 7 seconds
......@@ -14,7 +14,27 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
### Removed
## 2.4.0 - 2021-08-xx
## 2.4.1 - 2021-08-29
### Changed
- Make `mix pleroma.database set_text_search_config` run concurrently and indefinitely
### Added
- AdminAPI: Missing configuration description for StealEmojiPolicy
### Fixed
- MastodonAPI: Stream out Create activities
- MRF ObjectAgePolicy: Fix pattern matching on "published"
- TwitterAPI: Make `change_password` and `change_email` require params on body instead of query
- Subscription(Bell) Notifications: Don't create from Pipeline Ingested replies
- AdminAPI: Fix rendering reports containing a `nil` object
- Mastodon API: Activity Search fallbacks on status fetching after a DB Timeout/Error
- Mastodon API: Fix crash in Streamer related to reblogging
- AdminAPI: List available frontends when `static/frontends` folder is missing
- Make activity search properly use language-aware GIN indexes
- AdminAPI: Fix suggestions for MRF Policies
## 2.4.0 - 2021-08-08
### Changed
......
......@@ -5,7 +5,7 @@ Pleroma's full text search feature is powered by PostgreSQL's native [text searc
## Setup and test the new search config
In most cases, you would need an extension installed to support parsing CJK text. Here are a few extension you may choose from, or you are more than welcome to share additional ones you found working for you with the rest of Pleroma community.
In most cases, you would need an extension installed to support parsing CJK text. Here are a few extensions you may choose from, or you are more than welcome to share additional ones you found working for you with the rest of Pleroma community.
* [a generic n-gram parser](https://github.com/huangjimmy/pg_cjk_parser) supports Simplifed/Traditional Chinese, Japanese, and Korean
* [a Korean parser](https://github.com/i0seph/textsearch_ko) based on mecab
......@@ -34,7 +34,7 @@ Check output of the query, and see if it matches your expectation.
mix pleroma.database set_text_search_config YOUR.CONFIG
```
Note: index update may take a while.
Note: index update may take a while, and it can be done while the instance is up and running, so you may restart db connection as soon as you see `Recreate index` in task output.
## Restart database connection
Since some changes above will only apply with a new database connection, you will have to restart either Pleroma or PostgreSQL process, or use `pg_terminate_backend` SQL command without restarting either.
......
# Installing on Alpine Linux
{! backend/installation/otp_vs_from_source_source.include !}
## Installation
This guide is a step-by-step installation guide for Alpine Linux. The instructions were verified against Alpine v3.10 standard image. You might miss additional dependencies if you use `netboot` instead.
......
# Installing on Arch Linux
{! backend/installation/otp_vs_from_source_source.include !}
## Installation
This guide will assume that you have administrative rights, either as root or a user with [sudo permissions](https://wiki.archlinux.org/index.php/Sudo). If you want to run this guide with root, ignore the `sudo` at the beginning of the lines, unless it calls a user like `sudo -Hu pleroma`; in this case, use `su <username> -s $SHELL -c 'command'` instead.
......
# Installing on Debian Based Distributions
{! backend/installation/otp_vs_from_source_source.include !}
## Installation
This guide will assume you are on Debian 11 (“bullseye”) or later. This guide should also work with Ubuntu 18.04 (“Bionic Beaver”) and later. It also assumes that you have administrative rights, either as root or a user with [sudo permissions](https://www.digitalocean.com/community/tutorials/how-to-add-delete-and-grant-sudo-privileges-to-users-on-a-debian-vps). If you want to run this guide with root, ignore the `sudo` at the beginning of the lines, unless it calls a user like `sudo -Hu pleroma`; in this case, use `su <username> -s $SHELL -c 'command'` instead.
......
# Installing on Gentoo GNU/Linux
{! backend/installation/otp_vs_from_source_source.include !}
## Installation
This guide will assume that you have administrative rights, either as root or a user with [sudo permissions](https://wiki.gentoo.org/wiki/Sudo). Lines that begin with `#` indicate that they should be run as the superuser. Lines using `$` should be run as the indicated user, e.g. `pleroma$` should be run as the `pleroma` user.
......
# Switching a from-source install to OTP releases
## What are OTP releases?
OTP releases are as close as you can get to binary releases with Erlang/Elixir. The release is self-contained, and provides everything needed to boot it, it is easily administered via the provided shell script to open up a remote console, start/stop/restart the release, start in the background, send remote commands, and more.
{! backend/installation/otp_vs_from_source.include !}
In this guide we cover how you can migrate from a from source installation to one using OTP releases.
## Pre-requisites
You will be running commands as root. If you aren't root already, please elevate your priviledges by executing `sudo su`/`su`.
......
# Installing on Linux using OTP releases
{! backend/installation/otp_vs_from_source.include !}
This guide covers a installation using an OTP release. To install Pleroma from source, please check out the corresponding guide for your distro.
## Pre-requisites
* A machine running Linux with GNU (e.g. Debian, Ubuntu) or musl (e.g. Alpine) libc and `x86_64`, `aarch64` or `armv7l` CPU, you have root access to. If you are not sure if it's compatible see [Detecting flavour section](#detecting-flavour) below
* A (sub)domain pointed to the machine
......
## OTP releases vs from-source installations
There are two ways to install Pleroma. You can use OTP releases or do a from-source installation. OTP releases are as close as you can get to binary releases with Erlang/Elixir. The release is self-contained, and provides everything needed to boot it, it is easily administered via the provided shell script to open up a remote console, start/stop/restart the release, start in the background, send remote commands, and more. With from source installations you install Pleroma from source, meaning you have to install certain dependencies like Erlang+Elixir and compile Pleroma yourself.
{! backend/installation/otp_vs_from_source.include !}
This guide covers a from-source installation. To install using OTP releases, please check out [the OTP guide](./otp_en.md).
......@@ -209,7 +209,9 @@ def run(["set_text_search_config", tsconfig]) do
new.fts_content := to_tsvector(new.data->>'content');
RETURN new;
END
$$ LANGUAGE plpgsql"
$$ LANGUAGE plpgsql",
[],
timeout: :infinity
)
shell_info("Refresh RUM index")
......@@ -219,7 +221,9 @@ def run(["set_text_search_config", tsconfig]) do
Ecto.Adapters.SQL.query!(
Pleroma.Repo,
"CREATE INDEX objects_fts ON objects USING gin(to_tsvector('#{tsconfig}', data->>'content')); "
"CREATE INDEX CONCURRENTLY objects_fts ON objects USING gin(to_tsvector('#{tsconfig}', data->>'content')); ",
[],
timeout: :infinity
)
end
......
......@@ -26,19 +26,23 @@ def search(user, search_query, options \\ []) do
:plain
end
Activity
|> Activity.with_preloaded_object()
|> Activity.restrict_deactivated_users()
|> restrict_public()
|> query_with(index_type, search_query, search_function)
|> maybe_restrict_local(user)
|> maybe_restrict_author(author)
|> maybe_restrict_blocked(user)
|> Pagination.fetch_paginated(
%{"offset" => offset, "limit" => limit, "skip_order" => index_type == :rum},
:offset
)
|> maybe_fetch(user, search_query)
try do
Activity
|> Activity.with_preloaded_object()
|> Activity.restrict_deactivated_users()
|> restrict_public()
|> query_with(index_type, search_query, search_function)
|> maybe_restrict_local(user)
|> maybe_restrict_author(author)
|> maybe_restrict_blocked(user)
|> Pagination.fetch_paginated(
%{"offset" => offset, "limit" => limit, "skip_order" => index_type == :rum},
:offset
)
|> maybe_fetch(user, search_query)
rescue
_ -> maybe_fetch([], user, search_query)
end
end
def maybe_restrict_author(query, %User{} = author) do
......@@ -61,10 +65,17 @@ defp restrict_public(q) do
end
defp query_with(q, :gin, search_query, :plain) do
%{rows: [[tsc]]} =
Ecto.Adapters.SQL.query!(
Pleroma.Repo,
"select current_setting('default_text_search_config')::regconfig::oid;"
)
from([a, o] in q,
where:
fragment(
"to_tsvector(?->>'content') @@ plainto_tsquery(?)",
"to_tsvector(?::oid::regconfig, ?->>'content') @@ plainto_tsquery(?)",
^tsc,
o.data,
^search_query
)
......@@ -72,10 +83,17 @@ defp query_with(q, :gin, search_query, :plain) do
end
defp query_with(q, :gin, search_query, :websearch) do
%{rows: [[tsc]]} =
Ecto.Adapters.SQL.query!(
Pleroma.Repo,
"select current_setting('default_text_search_config')::regconfig::oid;"
)
from([a, o] in q,
where:
fragment(
"to_tsvector(?->>'content') @@ websearch_to_tsquery(?)",
"to_tsvector(?::oid::regconfig, ?->>'content') @@ websearch_to_tsquery(?)",
^tsc,
o.data,
^search_query
)
......
......@@ -21,7 +21,7 @@ defmodule Pleroma.Web.ActivityPub.MRF do
type: [:module, {:list, :module}],
description:
"A list of MRF policies enabled. Module names are shortened (removed leading `Pleroma.Web.ActivityPub.MRF.` part), but on adding custom module you need to use full name.",
suggestions: {:list_behaviour_implementations, Pleroma.Web.ActivityPub.MRF}
suggestions: {:list_behaviour_implementations, Pleroma.Web.ActivityPub.MRF.Policy}
},
%{
key: :transparency,
......
......@@ -49,6 +49,8 @@ defp check_delist(message, actions) do
message
|> Map.put("to", to)
|> Map.put("cc", cc)
|> Kernel.put_in(["object", "to"], to)
|> Kernel.put_in(["object", "cc"], cc)
{:ok, message}
else
......@@ -70,6 +72,8 @@ defp check_strip_followers(message, actions) do
message
|> Map.put("to", to)
|> Map.put("cc", cc)
|> Kernel.put_in(["object", "to"], to)
|> Kernel.put_in(["object", "cc"], cc)
{:ok, message}
else
......@@ -82,7 +86,7 @@ defp check_strip_followers(message, actions) do
end
@impl true
def filter(%{"type" => "Create", "published" => _} = message) do
def filter(%{"type" => "Create", "object" => %{"published" => _}} = message) do
with actions <- Config.get([:mrf_object_age, :actions]),
{:reject, _} <- check_date(message),
{:ok, message} <- check_reject(message, actions),
......
......@@ -92,6 +92,51 @@ def filter(%{"object" => %{"emoji" => foreign_emojis, "actor" => actor}} = messa
def filter(message), do: {:ok, message}
@impl true
@spec config_description :: %{
children: [
%{
description: <<_::272, _::_*256>>,
key: :hosts | :rejected_shortcodes | :size_limit,
suggestions: [any(), ...],
type: {:list, :string} | {:list, :string} | :integer
},
...
],
description: <<_::448>>,
key: :mrf_steal_emoji,
label: <<_::80>>,
related_policy: <<_::352>>
}
def config_description do
%{
key: :mrf_steal_emoji,
related_policy: "Pleroma.Web.ActivityPub.MRF.StealEmojiPolicy",
label: "MRF Emojis",
description: "Steals emojis from selected instances when it sees them.",
children: [
%{
key: :hosts,
type: {:list, :string},
description: "List of hosts to steal emojis from",
suggestions: [""]
},
%{
key: :rejected_shortcodes,
type: {:list, :string},
description: "Regex-list of shortcodes to reject",
suggestions: [""]
},
%{
key: :size_limit,
type: :integer,
description: "File size limit (in bytes), checked before an emoji is saved to the disk",
suggestions: ["100000"]
}
]
}
end
@impl true
def describe do
{:ok, %{}}
......
......@@ -10,7 +10,6 @@ defmodule Pleroma.Web.ActivityPub.SideEffects do
collection, and so on.
"""
alias Pleroma.Activity
alias Pleroma.Activity.Ir.Topics
alias Pleroma.Chat
alias Pleroma.Chat.MessageReference
alias Pleroma.FollowingRelationship
......@@ -225,6 +224,8 @@ def handle(%{data: %{"type" => "Create"}} = activity, meta) do
meta
|> add_notifications(notifications)
ap_streamer().stream_out(activity)
{:ok, activity, meta}
else
e -> Repo.rollback(e)
......@@ -245,9 +246,7 @@ def handle(%{data: %{"type" => "Announce"}} = object, meta) do
if !User.is_internal_user?(user) do
Notification.create_notifications(object)
object
|> Topics.get_activity_topics()
|> Streamer.stream(object)
ap_streamer().stream_out(object)
end
{:ok, object, meta}
......
......@@ -35,6 +35,12 @@ def install(%{body_params: params} = conn, _params) do
end
defp installed do
File.ls!(Pleroma.Frontend.dir())
frontend_directory = Pleroma.Frontend.dir()
if File.exists?(frontend_directory) do
File.ls!(frontend_directory)
else
[]
end
end
end
......@@ -13,7 +13,9 @@ def extract_report_info(
account = User.get_cached_by_ap_id(account_ap_id)
statuses =
Enum.map(status_ap_ids, fn
status_ap_ids
|> Enum.reject(&is_nil(&1))
|> Enum.map(fn
act when is_map(act) -> Activity.get_by_ap_id_with_object(act["id"])
act when is_binary(act) -> Activity.get_by_ap_id_with_object(act)
end)
......
......@@ -8,6 +8,8 @@ defmodule Pleroma.Web.ApiSpec.TwitterUtilOperation do
alias Pleroma.Web.ApiSpec.Schemas.ApiError
alias Pleroma.Web.ApiSpec.Schemas.BooleanLike
import Pleroma.Web.ApiSpec.Helpers
def open_api_operation(action) do
operation = String.to_existing_atom("#{action}_operation")
apply(__MODULE__, operation, [])
......@@ -63,17 +65,7 @@ def change_password_operation do
summary: "Change account password",
security: [%{"oAuth" => ["write:accounts"]}],
operationId: "UtilController.change_password",
parameters: [
Operation.parameter(:password, :query, :string, "Current password", required: true),
Operation.parameter(:new_password, :query, :string, "New password", required: true),
Operation.parameter(
:new_password_confirmation,
:query,
:string,
"New password, confirmation",
required: true
)
],
requestBody: request_body("Parameters", change_password_request(), required: true),
responses: %{
200 =>
Operation.response("Success", "application/json", %Schema{
......@@ -86,17 +78,30 @@ def change_password_operation do
}
end
defp change_password_request do
%Schema{
title: "ChangePasswordRequest",
description: "POST body for changing the account's passowrd",
type: :object,
required: [:password, :new_password, :new_password_confirmation],
properties: %{
password: %Schema{type: :string, description: "Current password"},
new_password: %Schema{type: :string, description: "New password"},
new_password_confirmation: %Schema{
type: :string,
description: "New password, confirmation"
}
}
}
end
def change_email_operation do
%Operation{
tags: ["Account credentials"],
summary: "Change account email",
security: [%{"oAuth" => ["write:accounts"]}],
operationId: "UtilController.change_email",
parameters: [
Operation.parameter(:password, :query, :string, "Current password", required: true),
Operation.parameter(:email, :query, :string, "New email", required: true)
],
requestBody: nil,
requestBody: request_body("Parameters", change_email_request(), required: true),
responses: %{
200 =>
Operation.response("Success", "application/json", %Schema{
......@@ -109,6 +114,19 @@ def change_email_operation do
}
end
defp change_email_request do
%Schema{
title: "ChangeEmailRequest",
description: "POST body for changing the account's email",
type: :object,
required: [:email, :password],
properties: %{
email: %Schema{type: :string, description: "New email"},
password: %Schema{type: :string, description: "Current password"}
}
}
end
def update_notificaton_settings_operation do
%Operation{
tags: ["Accounts"],
......
......@@ -412,19 +412,14 @@ def maybe_notify_mentioned_recipients(
def maybe_notify_mentioned_recipients(recipients, _), do: recipients
# Do not notify subscribers if author is making a reply
def maybe_notify_subscribers(recipients, %Activity{
object: %Object{data: %{"inReplyTo" => _ap_id}}
}) do
recipients
end
def maybe_notify_subscribers(
recipients,
%Activity{data: %{"actor" => actor, "type" => type}} = activity
)
when type == "Create" do
with %User{} = user <- User.get_cached_by_ap_id(actor) do
%Activity{data: %{"actor" => actor, "type" => "Create"}} = activity
) do
# Do not notify subscribers if author is making a reply
with %Object{data: object} <- Object.normalize(activity, fetch: false),
nil <- object["inReplyTo"],
%User{} = user <- User.get_cached_by_ap_id(actor) do
subscriber_ids =
user
|> User.subscriber_users()
......
......@@ -65,11 +65,19 @@ defp get_context_id(%{data: %{"context" => context}}) when is_binary(context),
defp get_context_id(_), do: nil
defp reblogged?(activity, user) do
object = Object.normalize(activity, fetch: false) || %{}
present?(user && user.ap_id in (object.data["announcements"] || []))
# Check if the user reblogged this status
defp reblogged?(activity, %User{ap_id: ap_id}) do
with %Object{data: %{"announcements" => announcements}} when is_list(announcements) <-
Object.normalize(activity, fetch: false) do
ap_id in announcements
else
_ -> false
end
end
# False if the user is logged out
defp reblogged?(_activity, _user), do: false
def render("index.json", opts) do
reading_user = opts[:for]
......
......@@ -81,17 +81,13 @@ def update_notificaton_settings(%{assigns: %{user: user}} = conn, params) do
end
end
def change_password(%{assigns: %{user: user}} = conn, %{
password: password,
new_password: new_password,
new_password_confirmation: new_password_confirmation
}) do
case CommonAPI.Utils.confirm_current_password(user, password) do
def change_password(%{assigns: %{user: user}, body_params: body_params} = conn, %{}) do
case CommonAPI.Utils.confirm_current_password(user, body_params.password) do
{:ok, user} ->
with {:ok, _user} <-
User.reset_password(user, %{
password: new_password,
password_confirmation: new_password_confirmation
password: body_params.new_password,
password_confirmation: body_params.new_password_confirmation
}) do
json(conn, %{status: "success"})
else
......@@ -108,10 +104,10 @@ def change_password(%{assigns: %{user: user}} = conn, %{
end
end
def change_email(%{assigns: %{user: user}} = conn, %{password: password, email: email}) do
case CommonAPI.Utils.confirm_current_password(user, password) do
def change_email(%{assigns: %{user: user}, body_params: body_params} = conn, %{}) do
case CommonAPI.Utils.confirm_current_password(user, body_params.password) do
{:ok, user} ->
with {:ok, _user} <- User.change_email(user, email) do
with {:ok, _user} <- User.change_email(user, body_params.email) do
json(conn, %{status: "success"})
else
{:error, changeset} ->
......
......@@ -4,7 +4,7 @@ defmodule Pleroma.Mixfile do
def project do
[
app: :pleroma,
version: version("2.4.0"),
version: version("2.4.1"),
elixir: "~> 1.9",
elixirc_paths: elixirc_paths(Mix.env()),
compilers: [:phoenix, :gettext] ++ Mix.compilers(),
......
......@@ -23,7 +23,7 @@ defmodule Pleroma.Docs.GeneratorTest do
key: :filters,
type: {:list, :module},
description: "",
suggestions: {:list_behaviour_implementations, Pleroma.Web.ActivityPub.MRF}
suggestions: {:list_behaviour_implementations, Pleroma.Web.ActivityPub.MRF.Policy}
},
%{
key: Pleroma.Upload,
......
......@@ -22,6 +22,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.ObjectAgePolicyTest do
defp get_old_message do
File.read!("test/fixtures/mastodon-post-activity.json")
|> Jason.decode!()
|> Map.drop(["published"])
end
defp get_new_message do
......
......@@ -42,6 +42,20 @@ test "it lists available frontends", %{conn: conn} do
refute Enum.any?(response, fn frontend -> frontend["installed"] == true end)
end
test "it lists available frontends when no frontend folder was created yet", %{conn: conn} do
File.rm_rf(@dir)
response =
conn
|> get("/api/pleroma/admin/frontends")
|> json_response_and_validate_schema(:ok)
assert Enum.map(response, & &1["name"]) ==
Enum.map(Config.get([:frontends, :available]), fn {_, map} -> map["name"] end)
refute Enum.any?(response, fn frontend -> frontend["installed"] == true end)
end
end
describe "POST /api/pleroma/admin/frontends/install" do
......
......@@ -261,11 +261,8 @@ test "without permissions", %{conn: conn} do
conn =
conn
|> assign(:token, nil)
|> post(
"/api/pleroma/change_email?#{
URI.encode_query(%{password: "hi", email: "test@test.com"})
}"
)
|> put_req_header("content-type", "multipart/form-data")
|> post("/api/pleroma/change_email", %{password: "hi", email: "test@test.com"})
assert json_response_and_validate_schema(conn, 403) == %{
"error" => "Insufficient permissions: write:accounts."
......@@ -274,12 +271,9 @@ test "without permissions", %{conn: conn} do
test "with proper permissions and invalid password", %{conn: conn} do
conn =
post(
conn,