Commit dd2b3a8d authored by lain's avatar lain
Browse files

Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into feature/expire-mutes

parents 8f5589cf 78cea445
Pipeline #32569 passed with stages
in 15 minutes and 53 seconds
......@@ -25,7 +25,7 @@
#
# If you create your own checks, you must specify the source files for
# them here, so they can be loaded by Credo before running the analysis.
requires: [],
requires: ["./test/credo/check/consistency/file_location.ex"],
#
# Credo automatically checks for updates, like e.g. Hex does.
# You can disable this behaviour below:
......@@ -71,7 +71,6 @@
# set this value to 0 (zero).
{Credo.Check.Design.TagTODO, exit_status: 0},
{Credo.Check.Design.TagFIXME, exit_status: 0},
{Credo.Check.Readability.FunctionNames},
{Credo.Check.Readability.LargeNumbers},
{Credo.Check.Readability.MaxLineLength, priority: :low, max_length: 100},
......@@ -91,7 +90,6 @@
{Credo.Check.Readability.VariableNames},
{Credo.Check.Readability.Semicolons},
{Credo.Check.Readability.SpaceAfterCommas},
{Credo.Check.Refactor.DoubleBooleanNegation},
{Credo.Check.Refactor.CondStatements},
{Credo.Check.Refactor.CyclomaticComplexity},
......@@ -102,7 +100,6 @@
{Credo.Check.Refactor.Nesting},
{Credo.Check.Refactor.PipeChainStart},
{Credo.Check.Refactor.UnlessWithElse},
{Credo.Check.Warning.BoolOperationOnSameValues},
{Credo.Check.Warning.IExPry},
{Credo.Check.Warning.IoInspect},
......@@ -131,6 +128,7 @@
# Custom checks can be created using `mix credo.gen.check`.
#
{Credo.Check.Consistency.FileLocation}
]
}
]
......
......@@ -27,6 +27,8 @@ erl_crash.dump
# variables.
/config/*.secret.exs
/config/generated_config.exs
/config/*.env
# Database setup file, some may forget to delete it
/config/setup_db.psql
......
......@@ -25,6 +25,8 @@ before_script:
- apt-get update && apt-get install -y cmake
- mix local.hex --force
- mix local.rebar --force
- apt-get -qq update
- apt-get install -y libmagic-dev
build:
stage: build
......@@ -59,7 +61,7 @@ unit-testing:
alias: postgres
command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
script:
- apt-get update && apt-get install -y libimage-exiftool-perl
- apt-get update && apt-get install -y libimage-exiftool-perl ffmpeg
- mix deps.get
- mix ecto.create
- mix ecto.migrate
......@@ -93,7 +95,7 @@ unit-testing-rum:
<<: *global_variables
RUM_ENABLED: "true"
script:
- apt-get update && apt-get install -y libimage-exiftool-perl
- apt-get update && apt-get install -y libimage-exiftool-perl ffmpeg
- mix deps.get
- mix ecto.create
- mix ecto.migrate
......@@ -196,7 +198,7 @@ amd64:
variables: &release-variables
MIX_ENV: prod
before_script: &before-release
- apt-get update && apt-get install -y cmake
- apt-get update && apt-get install -y cmake libmagic-dev
- echo "import Mix.Config" > config/prod.secret.exs
- mix local.hex --force
- mix local.rebar --force
......@@ -215,7 +217,7 @@ amd64-musl:
cache: *release-cache
variables: *release-variables
before_script: &before-release-musl
- apk add git gcc g++ musl-dev make cmake
- apk add git gcc g++ musl-dev make cmake file-dev
- echo "import Mix.Config" > config/prod.secret.exs
- mix local.hex --force
- mix local.rebar --force
......
......@@ -5,20 +5,47 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
## Unreleased
### Added
- Mix tasks for controlling user account confirmation status in bulk (`mix pleroma.user confirm_all` and `mix pleroma.user unconfirm_all`)
- Mix task for sending confirmation emails to all unconfirmed users (`mix pleroma.email send_confirmation_mails`)
- Mix task option for force-unfollowing relays
- Media preview proxy (requires `ffmpeg` and `ImageMagick` to be installed and media proxy to be enabled; see `:media_preview_proxy` config for more details).
- Pleroma API: Importing the mutes users from CSV files.
- Experimental websocket-based federation between Pleroma instances.
- Support pagination of blocks and mutes
- App metrics: ability to restrict access to specified IP whitelist.
- Account backup
- Configuration: Add `:instance, autofollowing_nicknames` setting to provide a way to make accounts automatically follow new users that register on the local Pleroma instance.
- Ability to view remote timelines, with ex. `/api/v1/timelines/public?instance=lain.com` and streams `public:remote` and `public:remote:media`
### Changed
- **Breaking** Requires `libmagic` (or `file`) to guess file types.
- **Breaking:** Pleroma Admin API: emoji packs and files routes changed.
- **Breaking:** Sensitive/NSFW statuses no longer disable link previews.
- **Breaking:** App metrics endpoint (`/api/pleroma/app_metrics`) is disabled by default, check `docs/API/prometheus.md` on enabling and configuring.
- Search: Users are now findable by their urls.
- Renamed `:await_up_timeout` in `:connections_pool` namespace to `:connect_timeout`, old name is deprecated.
- Renamed `:timeout` in `pools` namespace to `:recv_timeout`, old name is deprecated.
- The `discoverable` field in the `User` struct will now add a NOINDEX metatag to profile pages when false.
- Users with the `discoverable` field set to false will not show up in searches.
- Minimum lifetime for ephmeral activities changed to 10 minutes and made configurable (`:min_lifetime` option).
- Introduced optional dependencies on `ffmpeg`, `ImageMagick`, `exiftool` software packages. Please refer to `docs/installation/optional/media_graphics_packages.md`.
- Polls now always return a `voters_count`, even if they are single-choice
<details>
<summary>API Changes</summary>
### Added
- Media preview proxy (requires media proxy be enabled; see `:media_preview_proxy` config for more details).
- Pleroma API: Importing the mutes users from CSV files.
- Experimental websocket-based federation between Pleroma instances.
- Admin API: Importing emoji from a zip file
- User and conversation mutes can now auto-expire if `expires_in` parameter was given while adding the mute.
- Pleroma API: Pagination for remote/local packs and emoji.
- Admin API: (`GET /api/pleroma/admin/users`) added filters user by `unconfirmed` status
- Admin API: (`GET /api/pleroma/admin/users`) added filters user by `actor_type`
- Pleroma API: Add `idempotency_key` to the chat message entity that can be used for optimistic message sending.
- Pleroma API: (`GET /api/v1/pleroma/federation_status`) Add a way to get a list of unreachable instances.
- Mastodon API: User and conversation mutes can now auto-expire if `expires_in` parameter was given while adding the mute.
</details>
### Removed
......@@ -29,7 +56,19 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
switched to a new configuration mechanism, however it was not officially removed until now.
### Fixed
- Add documented-but-missing chat pagination.
- Allow sending out emails again.
- Allow sending chat messages to yourself.
- Fix remote users with a whitespace name.
- OStatus / static FE endpoints: fixed inaccessibility for anonymous users on non-federating instances, switched to handling per `:restrict_unauthenticated` setting.
- Mastodon API: Current user is now included in conversation if it's the only participant
- Mastodon API: Fixed last_status.account being not filled with account data
## Unreleased (Patch)
### Changed
- API: Empty parameter values for integer parameters are now ignored in non-strict validaton mode.
## [2.1.2] - 2020-09-17
......
......@@ -59,8 +59,6 @@
"BLH1qVhJItRGCfxgTtONfsOKDc9VRAraXw-3NsmjMngWSh7NxOizN6bkuRA7iLTMPS82PjwJAr3UoK9EC1IFrz4",
private_key: "_-XZ0iebPrRfZ_o0-IatTdszYa8VCH1yLN-JauK7HHA"
config :web_push_encryption, :http_client, Pleroma.Web.WebPushHttpClientMock
config :pleroma, Pleroma.ScheduledActivity,
daily_user_limit: 2,
total_user_limit: 3,
......
......@@ -123,7 +123,6 @@
# Configures the endpoint
config :pleroma, Pleroma.Web.Endpoint,
instrumenters: [Pleroma.Web.Endpoint.Instrumenter],
url: [host: "localhost"],
http: [
ip: {127, 0, 0, 1},
......@@ -143,7 +142,7 @@
secret_key_base: "aK4Abxf29xU9TTDKre9coZPUgevcVCFQJe/5xP/7Lt4BEif6idBIbjupVbOrbKxl",
signing_salt: "CqaoopA2",
render_errors: [view: Pleroma.Web.ErrorView, accepts: ~w(json)],
pubsub: [name: Pleroma.PubSub, adapter: Phoenix.PubSub.PG2],
pubsub_server: Pleroma.PubSub,
secure_cookie_flag: true,
extra_cookie_attrs: [
"SameSite=Lax"
......@@ -235,6 +234,7 @@
"text/bbcode"
],
autofollowed_nicknames: [],
autofollowing_nicknames: [],
max_pinned_statuses: 1,
attachment_links: false,
max_report_comment_size: 1000,
......@@ -551,6 +551,7 @@
queues: [
activity_expiration: 10,
token_expiration: 5,
backup: 1,
federator_incoming: 50,
federator_outgoing: 50,
ingestion_queue: 50,
......@@ -637,7 +638,12 @@
config :pleroma, Pleroma.Emails.NewUsersDigestEmail, enabled: false
config :prometheus, Pleroma.Web.Endpoint.MetricsExporter, path: "/api/pleroma/app_metrics"
config :prometheus, Pleroma.Web.Endpoint.MetricsExporter,
enabled: false,
auth: false,
ip_whitelist: [],
path: "/api/pleroma/app_metrics",
format: :text
config :pleroma, Pleroma.ScheduledActivity,
daily_user_limit: 25,
......@@ -678,7 +684,18 @@
config :pleroma, Pleroma.Workers.PurgeExpiredActivity, enabled: true, min_lifetime: 600
config :pleroma, Pleroma.Plugs.RemoteIp, enabled: true
config :pleroma, Pleroma.Web.Plugs.RemoteIp,
enabled: true,
headers: ["x-forwarded-for"],
proxies: [],
reserved: [
"127.0.0.0/8",
"::1/128",
"fc00::/7",
"10.0.0.0/8",
"172.16.0.0/12",
"192.168.0.0/16"
]
config :pleroma, :static_fe, enabled: false
......@@ -792,6 +809,8 @@
timeout: 300_000
]
config :pleroma, :majic_pool, size: 2
private_instance? = :if_instance_is_private
config :pleroma, :restrict_unauthenticated,
......@@ -810,7 +829,7 @@
config :ex_aws, http_client: Pleroma.HTTP.ExAws
config :web_push_encryption, http_client: Pleroma.HTTP
config :web_push_encryption, http_client: Pleroma.HTTP.WebPush
config :pleroma, :instances_favicons, enabled: false
......@@ -818,6 +837,11 @@
config :pleroma, Pleroma.Web.Auth.Authenticator, Pleroma.Web.Auth.PleromaAuthenticator
config :pleroma, Pleroma.User.Backup,
purge_after_days: 30,
limit_days: 7,
dir: nil
# Import environment specific config. This must remain at the bottom
# of this file so it overrides the configuration defined above.
import_config "#{Mix.env()}.exs"
......@@ -44,11 +44,13 @@
},
%{
key: "git",
label: "Git Repository URL",
type: :string,
description: "URL of the git repository of the frontend"
},
%{
key: "build_url",
label: "Build URL",
type: :string,
description:
"Either an url to a zip file containing the frontend or a template to build it by inserting the `ref`. The string `${ref}` will be replaced by the configured `ref`.",
......@@ -56,6 +58,7 @@
},
%{
key: "build_dir",
label: "Build directory",
type: :string,
description: "The directory inside the zip file "
}
......@@ -826,13 +829,13 @@
key: :autofollowed_nicknames,
type: {:list, :string},
description:
"Set to nicknames of (local) users that every new user should automatically follow",
suggestions: [
"lain",
"kaniini",
"lanodan",
"rinpatch"
]
"Set to nicknames of (local) users that every new user should automatically follow"
},
%{
key: :autofollowing_nicknames,
type: {:list, :string},
description:
"Set to nicknames of (local) users that automatically follows every newly registered user"
},
%{
key: :attachment_links,
......@@ -1754,28 +1757,37 @@
related_policy: "Pleroma.Web.ActivityPub.MRF.KeywordPolicy",
label: "MRF Keyword",
type: :group,
description: "Reject or Word-Replace messages with a keyword or regex",
description:
"Reject or Word-Replace messages matching a keyword or [Regex](https://hexdocs.pm/elixir/Regex.html).",
children: [
%{
key: :reject,
type: {:list, :string},
description:
"A list of patterns which result in message being rejected. Each pattern can be a string or a regular expression.",
description: """
A list of patterns which result in message being rejected.
Each pattern can be a string or [Regex](https://hexdocs.pm/elixir/Regex.html) in the format of `~r/PATTERN/`.
""",
suggestions: ["foo", ~r/foo/iu]
},
%{
key: :federated_timeline_removal,
type: {:list, :string},
description:
"A list of patterns which result in message being removed from federated timelines (a.k.a unlisted). Each pattern can be a string or a regular expression.",
description: """
A list of patterns which result in message being removed from federated timelines (a.k.a unlisted).
Each pattern can be a string or [Regex](https://hexdocs.pm/elixir/Regex.html) in the format of `~r/PATTERN/`.
""",
suggestions: ["foo", ~r/foo/iu]
},
%{
key: :replace,
type: {:list, :tuple},
description:
"A list of tuples containing {pattern, replacement}. Each pattern can be a string or a regular expression.",
suggestions: [{"foo", "bar"}, {~r/foo/iu, "bar"}]
description: """
**Pattern**: a string or [Regex](https://hexdocs.pm/elixir/Regex.html) in the format of `~r/PATTERN/`.
**Replacement**: a string. Leaving the field empty is permitted.
"""
}
]
},
......@@ -2285,6 +2297,12 @@
description: "Activity expiration queue",
suggestions: [10]
},
%{
key: :backup,
type: :integer,
description: "Backup queue",
suggestions: [1]
},
%{
key: :attachments_cleanup,
type: :integer,
......@@ -3247,10 +3265,10 @@
},
%{
group: :pleroma,
key: Pleroma.Plugs.RemoteIp,
key: Pleroma.Web.Plugs.RemoteIp,
type: :group,
description: """
`Pleroma.Plugs.RemoteIp` is a shim to call [`RemoteIp`](https://git.pleroma.social/pleroma/remote_ip) but with runtime configuration.
`Pleroma.Web.Plugs.RemoteIp` is a shim to call [`RemoteIp`](https://git.pleroma.social/pleroma/remote_ip) but with runtime configuration.
**If your instance is not behind at least one reverse proxy, you should not enable this plug.**
""",
children: [
......@@ -3262,20 +3280,22 @@
%{
key: :headers,
type: {:list, :string},
description:
"A list of strings naming the `req_headers` to use when deriving the `remote_ip`. Order does not matter. Default: `~w[forwarded x-forwarded-for x-client-ip x-real-ip]`."
description: """
A list of strings naming the HTTP headers to use when deriving the true client IP. Default: `["x-forwarded-for"]`.
"""
},
%{
key: :proxies,
type: {:list, :string},
description:
"A list of strings in [CIDR](https://en.wikipedia.org/wiki/CIDR) notation specifying the IPs of known proxies. Default: `[]`."
"A list of upstream proxy IP subnets in CIDR notation from which we will parse the content of `headers`. Defaults to `[]`. IPv4 entries without a bitmask will be assumed to be /32 and IPv6 /128."
},
%{
key: :reserved,
type: {:list, :string},
description:
"Defaults to [localhost](https://en.wikipedia.org/wiki/Localhost) and [private network](https://en.wikipedia.org/wiki/Private_network)."
description: """
A list of reserved IP subnets in CIDR notation which should be ignored if found in `headers`. Defaults to `["127.0.0.0/8", "::1/128", "fc00::/7", "10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16"]`
"""
}
]
},
......@@ -3681,9 +3701,7 @@
type: :map,
description:
"A map containing available frontends and parameters for their installation.",
children: [
frontend_options
]
children: frontend_options
}
]
},
......@@ -3705,5 +3723,76 @@
]
}
]
},
%{
group: :pleroma,
key: :majic_pool,
type: :group,
description: "Majic/libmagic configuration",
children: [
%{
key: :size,
type: :integer,
description: "Number of majic workers to start.",
suggestions: [2]
}
]
},
%{
group: :pleroma,
key: Pleroma.User.Backup,
type: :group,
description: "Account Backup",
children: [
%{
key: :purge_after_days,
type: :integer,
description: "Remove backup achives after N days",
suggestions: [30]
},
%{
key: :limit_days,
type: :integer,
description: "Limit user to export not more often than once per N days",
suggestions: [7]
}
]
},
%{
group: :prometheus,
key: Pleroma.Web.Endpoint.MetricsExporter,
type: :group,
description: "Prometheus app metrics endpoint configuration",
children: [
%{
key: :enabled,
type: :boolean,
description: "[Pleroma extension] Enables app metrics endpoint."
},
%{
key: :ip_whitelist,
type: [{:list, :string}, {:list, :charlist}, {:list, :tuple}],
description:
"[Pleroma extension] If non-empty, restricts access to app metrics endpoint to specified IP addresses."
},
%{
key: :auth,
type: [:boolean, :tuple],
description: "Enables HTTP Basic Auth for app metrics endpoint.",
suggestion: [false, {:basic, "myusername", "mypassword"}]
},
%{
key: :path,
type: :string,
description: "App metrics endpoint URI path.",
suggestions: ["/api/pleroma/app_metrics"]
},
%{
key: :format,
type: :atom,
description: "App metrics endpoint output format.",
suggestions: [:text, :protobuf]
}
]
}
]
......@@ -83,8 +83,6 @@
"BLH1qVhJItRGCfxgTtONfsOKDc9VRAraXw-3NsmjMngWSh7NxOizN6bkuRA7iLTMPS82PjwJAr3UoK9EC1IFrz4",
private_key: "_-XZ0iebPrRfZ_o0-IatTdszYa8VCH1yLN-JauK7HHA"
config :web_push_encryption, :http_client, Pleroma.Web.WebPushHttpClientMock
config :pleroma, Oban,
queues: false,
crontab: false,
......@@ -115,7 +113,7 @@
config :pleroma, Pleroma.Emails.NewUsersDigestEmail, enabled: true
config :pleroma, Pleroma.Plugs.RemoteIp, enabled: false
config :pleroma, Pleroma.Web.Plugs.RemoteIp, enabled: false
config :pleroma, Pleroma.Web.ApiSpec.CastAndValidate, strict: true
......
{
"skip_files": [
"test/support",
"lib/mix/tasks/pleroma/benchmark.ex"
"lib/mix/tasks/pleroma/benchmark.ex",
"lib/credo/check/consistency/file_location.ex"
]
}
\ No newline at end of file
......@@ -20,12 +20,14 @@ Configuration options:
- `external`: only external users
- `active`: only active users
- `need_approval`: only unapproved users
- `unconfirmed`: only unconfirmed users
- `deactivated`: only deactivated users
- `is_admin`: users with admin role
- `is_moderator`: users with moderator role
- *optional* `page`: **integer** page number
- *optional* `page_size`: **integer** number of users per page (default is `50`)
- *optional* `tags`: **[string]** tags list
- *optional* `actor_types`: **[string]** actor type list (`Person`, `Service`, `Application`)
- *optional* `name`: **string** user display name
- *optional* `email`: **string** user email
- Example: `https://mypleroma.org/api/pleroma/admin/users?query=john&filters=local,active&page=1&page_size=10&tags[]=some_tag&tags[]=another_tag&name=display_name&email=email@example.com`
......@@ -349,9 +351,9 @@ Response:
### Unfollow a Relay
Params:
* `relay_url`
- Params:
- `relay_url`
- *optional* `force`: forcefully unfollow a relay even when the relay is not available. (default is `false`)
Response:
......
......@@ -116,6 +116,10 @@ The modified chat message
This will return a list of chats that you have been involved in, sorted by their
last update (so new chats will be at the top).
Parameters:
- with_muted: Include chats from muted users (boolean).
Returned data:
```json
......@@ -173,11 +177,14 @@ Returned data:
"created_at": "2020-04-21T15:06:45.000Z",
"emojis": [],
"id": "12",
"unread": false
"unread": false,
"idempotency_key": "75442486-0874-440c-9db1-a7006c25a31f"
}
]
```
- idempotency_key: The copy of the `idempotency-key` HTTP request header that can be used for optimistic message sending. Included only during the first few minutes after the message creation.
### Posting a chat message
Posting a chat message for given Chat id works like this:
......
......@@ -9,9 +9,13 @@ Pleroma uses 128-bit ids as opposed to Mastodon's 64 bits. However just like Mas
## Timelines
Adding the parameter `with_muted=true` to the timeline queries will also return activities by muted (not by blocked!) users.
Adding the parameter `exclude_visibilities` to the timeline queries will exclude the statuses with the given visibilities. The parameter accepts an array of visibility types (`public`, `unlisted`, `private`, `direct`), e.g., `exclude_visibilities[]=direct&exclude_visibilities[]=private`.
Adding the parameter `reply_visibility` to the public and home timelines queries will filter replies. Possible values: without parameter (default) shows all replies, `following` - replies directed to you or users you follow, `self` - replies directed to you.
Adding the parameter `instance=lain.com` to the public timeline will show only statuses originating from `lain.com` (or any remote instance).
## Statuses
- `visibility`: has an additional possible value `list`
......@@ -249,6 +253,8 @@ Has these additional fields under the `pleroma` object:
There is an additional `user:pleroma_chat` stream. Incoming chat messages will make the current chat be sent to this `user` stream. The `event` of an incoming chat message is `pleroma:chat_update`. The payload is the updated chat with the incoming chat message in the `last_message` field.
For viewing remote server timelines, there are `public:remote` and `public:remote:media` streams. Each of these accept a parameter like `?instance=lain.com`.
## Not implemented
Pleroma is generally compatible with the Mastodon 2.7.2 API, but some newer features and non-essential features are omitted. These features usually return an HTTP 200 status code, but with an empty response. While they may be added in the future, they are considered low priority.
......
......@@ -378,44 +378,43 @@ The status posting endpoint takes an additional parameter, `in_reply_to_conversa
* Params: None
* Response: JSON, returns a list of Mastodon Conversation entities that were marked as read (200 - healthy, 503 unhealthy).
## `GET /api/pleroma/emoji/packs/import`
### Imports packs from filesystem
* Method `GET`
* Authentication: required
* Params: None
* Response: JSON, returns a list of imported packs.
## `GET /api/pleroma/emoji/pack?name=:name`
### Get pack.json for the pack
## `GET /api/pleroma/emoji/packs/remote`
### Make request to another instance for packs list
* Method `GET`
* Authentication: required
* Authentication: not required
* Params:
* `url`: url of the instance to get packs from
* Response: JSON with the pack list, hashmap with pack name and pack contents
* `page`: page number for files (default 1)
* `page_size`: page size for files (default 30)
* Response: JSON, pack json with `files`, `files_count` and `pack` keys with 200 status or 404 if the pack does not exist.
## `POST /api/pleroma/emoji/packs/download`
### Download pack from another instance
* Method `POST`
* Authentication: required