Commit 30e9b22f authored by lain's avatar lain
Browse files

Merge branch 'develop' into feature/activitypub

parents 6268b7e0 d5a13c10
image: elixir:1.5
services:
- postgres:9.6.2
variables:
POSTGRES_DB: pleroma_test
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
stages:
- test
before_script:
- mix local.hex --force
- mix local.rebar --force
- mix deps.get
- MIX_ENV=test mix ecto.create
- MIX_ENV=test mix ecto.migrate
unit-testing:
stage: test
script:
- MIX_ENV=test mix test
......@@ -4,76 +4,36 @@
Pleroma is an OStatus-compatible social networking server written in Elixir, compatible with GNU Social and Mastodon. It is high-performance and can run on small devices like a Raspberry Pi.
For clients it supports both the GNU Social API with Qvitter extensions and the Mastodon client API.
For clients it supports both the [GNU Social API with Qvitter extensions](https://twitter-api.readthedocs.io/en/latest/index.html) and the [Mastodon client API](https://github.com/tootsuite/documentation/blob/master/Using-the-API/API.md).
Mobile clients that are known to work:
Mobile clients that are known to work well:
* Twidere
* Tusky
* Pawoo (Android)
* Pawoo (Android + iOS)
* Subway Tooter
* Amaroq (iOS)
* Tootdon (Android + iOS)
No release has been made yet, but several servers have been online for months already. If you want to run your own server, feel free to contact us at @lain@pleroma.soykaf.com or in our dev chat at https://matrix.heldscal.la/#/room/#pleromafe:matrix.heldscal.la.
No release has been made yet, but several servers have been online for months already. If you want to run your own server, feel free to contact us at @lain@pleroma.soykaf.com or in our dev chat at #pleroma on freenode or via matrix at https://matrix.heldscal.la/#/room/#freenode_#pleroma:matrix.org.
## Installation
### Dependencies
* Postgresql version 9.6 or newer
* Elixir version 1.4 or newer
* Elixir version 1.4 or newer (you will also need erlang-dev, erlang-parsetools, erlang-xmerl packages)
* Build-essential tools
#### Installing dependencies on Debian system
PostgreSQL 9.6 should be available on Debian stable (Jessie) from "main" area. Install it using apt: `apt install postgresql-9.6`. Make sure that older versions are not installed since Debian allows multiple versions to coexist but still runs only one version.
### Configuration
You must install elixir 1.4+ from elixir-lang.org, because Debian repos only have 1.3.x version. You will need to add apt repo to sources.list(.d) and import GPG key. Follow instructions here: https://elixir-lang.org/install.html#unix-and-unix-like (See "Ubuntu or Debian 7"). This should be valid until Debian updates elixir in their repositories. Package you want is named `elixir`, so install it using `apt install elixir`
* Run `mix deps.get` to install elixir dependencies.
Elixir will also require `make` and probably other related software for building dependencies - in case you don't have them, get them via `apt install build-essential`
* Run `mix generate_config`. This will ask you a few questions about your instance and generate a configuration file in `config/generated_config.exs`. Check that and copy it to either `config/dev.secret.exs` or `config/prod.secret.exs`. It will also create a `config/setup_db.psql`, which you need to run as PostgreSQL superuser (i.e. `sudo su postgres -c "psql -f config/setup_db.psql"`). It will setup a pleroma db user, database and will setup needed extensions that need to be set up once as superuser.
### Preparation
* Run `mix ecto.migrate` to run the database migrations. You will have to do this again after certain updates.
* You probably want application to run as separte user - so create a new one: `adduser pleroma`, you can login as it via `su pleroma`
* Clone the git repository into new user's dir (clone as the pleroma user to avoid permissions errors)
* Again, as new user, install dependencies with `mix deps.get` if it asks you to install "hex" - agree to that.
### Database setup
* Create a database user and database for pleroma
* Open psql shell as postgres user: (as root) `su postgres -c psql`
* Create a new PostgreSQL user:
```sql
\c pleroma_dev
CREATE user pleroma;
ALTER user pleroma with encrypted password '<your password>';
GRANT ALL ON ALL tables IN SCHEMA public TO pleroma;
GRANT ALL ON ALL sequences IN SCHEMA public TO pleroma;
```
* Create `config/dev.secret.exs` and copy the database settings from `dev.exs` there.
* Change password in `config/dev.secret.exs`, and change user to `"pleroma"` (line like `username: "postgres"`)
* Create and update your database with `mix ecto.create && mix ecto.migrate`.
### Some additional configuration
* You will need to let pleroma instance to know what hostname/url it's running on. _THIS IS THE MOST IMPORTANT STEP. GET THIS WRONG AND YOU'LL HAVE TO RESET YOUR DATABASE_.
Create the file `config/dev.secret.exs`, add these lines at the end of the file:
```elixir
config :pleroma, Pleroma.Web.Endpoint,
url: [host: "example.tld", scheme: "https", port: 443]
```
replacing `example.tld` with your (sub)domain
* You should also setup your site name and admin email address. Look at config.exs for more available options.
```elixir
config :pleroma, :instance,
name: "My great instance",
email: "someone@example.com"
```
* You can check if your instance is configured correctly by running it with `mix phx.server` and checking the instance info endpoint at `/api/v1/instance`. If it shows your uri, name and email correctly, you are configured correctly. If it shows something like `localhost:4000`, your configuration is probably wrong, unless you are running a local development setup.
* The common and convenient way for adding HTTPS is by using Nginx as a reverse proxy. You can look at example Nginx configuration in `installation/pleroma.nginx`. If you need TLS/SSL certificates for HTTPS, you can look get some for free with letsencrypt: https://letsencrypt.org/
On Debian you can use `certbot` package and command to manage letsencrypt certificates.
......
......@@ -49,5 +49,5 @@
try do
import_config "dev.secret.exs"
rescue
_-> nil
_-> IO.puts("!!! RUNNING IN LOCALHOST DEV MODE! !!!\nFEDERATION WON'T WORK UNTIL YOU CONFIGURE A dev.secret.exs")
end
......@@ -18,7 +18,7 @@
username: "postgres",
password: "postgres",
database: "pleroma_test",
hostname: "localhost",
hostname: System.get_env("DB_HOST") || "localhost",
pool: Ecto.Adapters.SQL.Sandbox
......
......@@ -19,6 +19,9 @@ server {
server_name example.tld;
location / {
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_pass http://localhost:4000;
}
include snippets/well-known.conf;
......
defmodule Mix.Tasks.GenerateConfig do
use Mix.Task
@shortdoc "Generates a new config"
def run(_) do
IO.puts("Answer a few questions to generate a new config\n")
IO.puts("--- THIS WILL OVERWRITE YOUR config/generated_config.exs! ---\n")
domain = IO.gets("What is your domain name? (e.g. pleroma.soykaf.com): ") |> String.trim
name = IO.gets("What is the name of your instance? (e.g. Pleroma/Soykaf): ") |> String.trim
email = IO.gets("What's your admin email address: ") |> String.trim
secret = :crypto.strong_rand_bytes(64) |> Base.encode64 |> binary_part(0, 64)
dbpass = :crypto.strong_rand_bytes(64) |> Base.encode64 |> binary_part(0, 64)
resultSql = EEx.eval_file("lib/mix/tasks/sample_psql.eex", [dbpass: dbpass])
result = EEx.eval_file("lib/mix/tasks/sample_config.eex", [domain: domain, email: email, name: name, secret: secret, dbpass: dbpass])
IO.puts("\nWriting config to config/generated_config.exs.\n\nCheck it and configure your database, then copy it to either config/dev.secret.exs or config/prod.secret.exs")
File.write("config/generated_config.exs", result)
IO.puts("\nWriting setup_db.psql, please run it as postgre superuser, i.e.: sudo su postgres -c 'psql -f config/setup_db.psql'")
File.write("config/setup_db.psql", resultSql)
end
end
use Mix.Config
config :pleroma, Pleroma.Web.Endpoint,
url: [host: "<%= domain %>", scheme: "https", port: 443],
secret_key_base: "<%= secret %>"
config :pleroma, :instance,
name: "<%= name %>",
email: "<%= email %>",
limit: 5000,
registrations_open: true
# Configure your database
config :pleroma, Pleroma.Repo,
adapter: Ecto.Adapters.Postgres,
username: "pleroma",
password: "<%= dbpass %>",
database: "pleroma_dev",
hostname: "localhost",
pool_size: 10
CREATE USER pleroma WITH ENCRYPTED PASSWORD '<%= dbpass %>' CREATEDB;
-- in case someone runs this second time accidentally
ALTER USER pleroma WITH ENCRYPTED PASSWORD '<%= dbpass %>' CREATEDB;
CREATE DATABASE pleroma_dev;
ALTER DATABASE pleroma_dev OWNER TO pleroma;
\c pleroma_dev;
--Extensions made by ecto.migrate that need superuser access
CREATE EXTENSION IF NOT EXISTS citext;
defmodule Pleroma.PasswordResetToken do
use Ecto.Schema
import Ecto.Changeset
alias Pleroma.{User, PasswordResetToken, Repo}
schema "password_reset_tokens" do
belongs_to :user, User
field :token, :string
field :used, :boolean, default: false
timestamps()
end
def create_token(%User{} = user) do
token = :crypto.strong_rand_bytes(32) |> Base.url_encode64
token = %PasswordResetToken{
user_id: user.id,
used: false,
token: token
}
Repo.insert(token)
end
def used_changeset(struct) do
struct
|> cast(%{}, [])
|> put_change(:used, true)
end
def reset_password(token, data) do
with %{used: false} = token <- Repo.get_by(PasswordResetToken, %{token: token}),
%User{} = user <- Repo.get(User, token.user_id),
{:ok, _user} <- User.reset_password(user, data),
{:ok, token} <- Repo.update(used_changeset(token)) do
{:ok, token}
else
_e -> {:error, token}
end
end
end
......@@ -6,7 +6,8 @@ defmodule Pleroma.Activity do
schema "activities" do
field :data, :map
field :local, :boolean, default: true
has_many :notifications, Notification
field :actor, :string
has_many :notifications, Notification, on_delete: :delete_all
timestamps()
end
......@@ -16,24 +17,29 @@ def get_by_ap_id(ap_id) do
where: fragment("(?)->>'id' = ?", activity.data, ^to_string(ap_id)))
end
# TODO:
# Go through these and fix them everywhere.
# Wrong name, only returns create activities
def all_by_object_ap_id_q(ap_id) do
from activity in Activity,
where: fragment("(?)->'object'->>'id' = ?", activity.data, ^to_string(ap_id))
where: fragment("coalesce((?)->'object'->>'id', (?)->>'object') = ?", activity.data, activity.data, ^to_string(ap_id)),
where: fragment("(?)->>'type' = 'Create'", activity.data)
end
# Wrong name, returns all.
def all_non_create_by_object_ap_id_q(ap_id) do
from activity in Activity,
where: fragment("(?)->>'object' = ?", activity.data, ^to_string(ap_id))
where: fragment("coalesce((?)->'object'->>'id', (?)->>'object') = ?", activity.data, activity.data, ^to_string(ap_id))
end
# Wrong name plz fix thx
def all_by_object_ap_id(ap_id) do
Repo.all(all_by_object_ap_id_q(ap_id))
end
def get_create_activity_by_object_ap_id(ap_id) do
Repo.one(from activity in Activity,
where: fragment("(?)->'object'->>'id' = ?", activity.data, ^to_string(ap_id))
and fragment("(?)->>'type' = 'Create'", activity.data))
where: fragment("coalesce((?)->'object'->>'id', (?)->>'object') = ?", activity.data, activity.data, ^to_string(ap_id)),
where: fragment("(?)->>'type' = 'Create'", activity.data))
end
end
......@@ -19,8 +19,10 @@ def start(_type, _args) do
ttl_interval: 1000,
limit: 2500
]]),
worker(Pleroma.Web.Federator, [])
worker(Pleroma.Web.Federator, []),
worker(Pleroma.Web.ChatChannel.ChatChannelState, []),
]
++ if Mix.env == :test, do: [], else: [worker(Pleroma.Web.Streamer, [])]
# See http://elixir-lang.org/docs/stable/elixir/Supervisor.html
# for other strategies and supported options
......
defmodule Pleroma.Formatter do
alias Pleroma.User
@link_regex ~r/https?:\/\/[\w\.\/?=\-#%&]+[\w]/u
@link_regex ~r/https?:\/\/[\w\.\/?=\-#%&@~\(\)]+[\w\/]/u
def linkify(text) do
Regex.replace(@link_regex, text, "<a href='\\0'>\\0</a>")
end
@tag_regex ~r/\#\w+/u
def parse_tags(text) do
def parse_tags(text, data \\ %{}) do
Regex.scan(@tag_regex, text)
|> Enum.map(fn (["#" <> tag = full_tag]) -> {full_tag, String.downcase(tag)} end)
|> (fn map -> if data["sensitive"], do: [{"#nsfw", "nsfw"}] ++ map, else: map end).()
end
def parse_mentions(text) do
......@@ -23,6 +24,15 @@ def parse_mentions(text) do
|> Enum.filter(fn ({_match, user}) -> user end)
end
def html_escape(text) do
Regex.split(@link_regex, text, include_captures: true)
|> Enum.map_every(2, fn chunk ->
{:safe, part} = Phoenix.HTML.html_escape(chunk)
part
end)
|> Enum.join("")
end
@finmoji [
"a_trusted_friend",
"alandislands",
......@@ -122,4 +132,8 @@ def emojify(text, additional \\ nil) do
def get_emoji(text) do
Enum.filter(@emoji, fn ({emoji, _}) -> String.contains?(text, ":#{emoji}:") end)
end
def get_custom_emoji() do
@emoji
end
end
......@@ -36,7 +36,38 @@ def for_user(user, opts \\ %{}) do
Repo.all(query)
end
def create_notifications(%Activity{id: id, data: %{"to" => to, "type" => type}} = activity) when type in ["Create", "Like", "Announce", "Follow"] do
def get(%{id: user_id} = _user, id) do
query = from n in Notification,
where: n.id == ^id,
preload: [:activity]
notification = Repo.one(query)
case notification do
%{user_id: ^user_id} ->
{:ok, notification}
_ ->
{:error, "Cannot get notification"}
end
end
def clear(user) do
query = from n in Notification,
where: n.user_id == ^user.id
Repo.delete_all(query)
end
def dismiss(%{id: user_id} = _user, id) do
notification = Repo.get(Notification, id)
case notification do
%{user_id: ^user_id} ->
Repo.delete(notification)
_ ->
{:error, "Cannot dismiss notification"}
end
end
def create_notifications(%Activity{id: _, data: %{"to" => _, "type" => type}} = activity) when type in ["Create", "Like", "Announce", "Follow"] do
users = User.get_notified_from_activity(activity)
notifications = Enum.map(users, fn (user) -> create_notification(activity, user) end)
......@@ -46,9 +77,12 @@ def create_notifications(_), do: {:ok, []}
# TODO move to sql, too.
def create_notification(%Activity{} = activity, %User{} = user) do
notification = %Notification{user_id: user.id, activity_id: activity.id}
{:ok, notification} = Repo.insert(notification)
notification
unless User.blocks?(user, %{ap_id: activity.data["actor"]}) do
notification = %Notification{user_id: user.id, activity: activity}
{:ok, notification} = Repo.insert(notification)
Pleroma.Web.Streamer.stream("user", notification)
notification
end
end
end
......@@ -15,15 +15,16 @@ def create(data) do
end
def change(struct, params \\ %{}) do
changeset = struct
struct
|> cast(params, [:data])
|> validate_required([:data])
|> unique_constraint(:ap_id, name: :objects_unique_apid_index)
end
def get_by_ap_id(nil), do: nil
def get_by_ap_id(ap_id) do
Repo.one(from object in Object,
where: fragment("? @> ?", object.data, ^%{id: ap_id}))
where: fragment("(?)->>'id' = ?", object.data, ^ap_id))
end
def get_cached_by_ap_id(ap_id) do
......
......@@ -12,6 +12,7 @@ def call(%{assigns: %{user: %User{}}} = conn, _), do: conn
def call(conn, opts) do
with {:ok, username, password} <- decode_header(conn),
{:ok, user} <- opts[:fetcher].(username),
false <- !!user.info["deactivated"],
saved_user_id <- get_session(conn, :user_id),
{:ok, verified_user} <- verify(user, password, saved_user_id)
do
......@@ -44,7 +45,7 @@ defp verify(user, password, _user_id) do
defp decode_header(conn) do
with ["Basic " <> header] <- get_req_header(conn, "authorization"),
{:ok, userinfo} <- Base.decode64(header),
[username, password] <- String.split(userinfo, ":")
[username, password] <- String.split(userinfo, ":", parts: 2)
do
{:ok, username, password}
end
......
......@@ -9,10 +9,15 @@ def init(options) do
end
def call(%{assigns: %{user: %User{}}} = conn, _), do: conn
def call(conn, opts) do
with ["Bearer " <> header] <- get_req_header(conn, "authorization"),
%Token{user_id: user_id} <- Repo.get_by(Token, token: header),
%User{} = user <- Repo.get(User, user_id) do
def call(conn, _) do
token = case get_req_header(conn, "authorization") do
["Bearer " <> header] -> header
_ -> get_session(conn, :oauth_token)
end
with token when not is_nil(token) <- token,
%Token{user_id: user_id} <- Repo.get_by(Token, token: token),
%User{} = user <- Repo.get(User, user_id),
false <- !!user.info["deactivated"] do
conn
|> assign(:user, user)
else
......
......@@ -8,11 +8,18 @@ def store(%Plug.Upload{} = file) do
result_file = Path.join(upload_folder, file.filename)
File.cp!(file.path, result_file)
# fix content type on some image uploads
content_type = if file.content_type == "application/octet-stream" do
get_content_type(file.path)
else
file.content_type
end
%{
"type" => "Image",
"url" => [%{
"type" => "Link",
"mediaType" => file.content_type,
"mediaType" => content_type,
"href" => url_for(Path.join(uuid, :cow_uri.urlencode(file.filename)))
}],
"name" => file.filename,
......@@ -53,4 +60,34 @@ defp upload_path do
defp url_for(file) do
"#{Web.base_url()}/media/#{file}"
end
def get_content_type(file) do
match = File.open(file, [:read], fn(f) ->
case IO.binread(f, 8) do
<<0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a>> ->
"image/png"
<<0x47, 0x49, 0x46, 0x38, _, 0x61, _, _>> ->
"image/gif"
<<0xff, 0xd8, 0xff, _, _, _, _, _>> ->
"image/jpeg"
<<0x1a, 0x45, 0xdf, 0xa3, _, _, _, _>> ->
"video/webm"
<<0x00, 0x00, 0x00, _, 0x66, 0x74, 0x79, 0x70>> ->
"video/mp4"
<<0x49, 0x44, 0x33, _, _, _, _, _>> ->
"audio/mpeg"
<<0x4f, 0x67, 0x67, 0x53, 0x00, 0x02, 0x00, 0x00>> ->
"audio/ogg"
<<0x52, 0x49, 0x46, 0x46, _, _, _, _>> ->
"audio/wav"
_ ->
"application/octet-stream"
end
end)
case match do
{:ok, type} -> type
_e -> "application/octet-stream"
end
end
end
......@@ -5,8 +5,7 @@ defmodule Pleroma.User do
alias Pleroma.{Repo, User, Object, Web, Activity, Notification}
alias Comeonin.Pbkdf2
alias Pleroma.Web.{OStatus, Websub}
alias Pleroma.Web.ActivityPub.ActivityPub
alias Pleroma.Web.ActivityPub.Utils
alias Pleroma.Web.ActivityPub.{Utils, ActivityPub}
schema "users" do
field :bio, :string
......@@ -62,8 +61,9 @@ def info_changeset(struct, params \\ %{}) do
end
def user_info(%User{} = user) do
oneself = if user.local, do: 1, else: 0
%{
following_count: length(user.following),
following_count: length(user.following) - oneself,
note_count: user.info["note_count"] || 0,
follower_count: user.info["follower_count"] || 0
}
......@@ -89,7 +89,7 @@ def remote_user_creation(params) do
end
def update_changeset(struct, params \\ %{}) do
changeset = struct
struct
|> cast(params, [:bio, :name])
|> unique_constraint(:nickname)
|> validate_format(:nickname, ~r/^[a-zA-Z\d]+$/)
......@@ -97,6 +97,25 @@ def update_changeset(struct, params \\ %{}) do
|> validate_length(:name, min: 1, max: 100)
end
def password_update_changeset(struct, params) do
changeset = struct
|> cast(params, [:password, :password_confirmation])
|> validate_required([:password, :password_confirmation])
|> validate_confirmation(:password)
if changeset.valid? do
hashed = Pbkdf2.hashpwsalt(changeset.changes[:password])
changeset
|> put_change(:password_hash, hashed)
else
changeset
end
end
def reset_password(user, data) do
update_and_set_cache(password_update_changeset(user, data))
end
def register_changeset(struct, params \\ %{}) do
changeset = struct
|> cast(params, [:bio, :email, :name, :nickname, :password, :password_confirmation])
......@@ -123,9 +142,9 @@ def register_changeset(struct, params \\ %{}) do
end
end
def follow(%User{} = follower, %User{} = followed) do
def follow(%User{} = follower, %User{info: info} = followed) do
ap_followers = followed.follower_address
if following?(follower, followed) do
if following?(follower, followed) or info["deactivated"] do
{:error,
"Could not follow user: #{followed.nickname} is already on your list."}
else
......@@ -138,9 +157,9 @@ def follow(%User{} = follower, %User{} = followed) do
follower = follower
|> follow_changeset(%{following: following})
|> Repo.update
|> update_and_set_cache
{:ok, followed} = update_follower_count(followed)
{:ok, _} = update_follower_count(followed)
follower
end
......@@ -148,13 +167,13 @@ def follow(%User{} = follower, %User{} = followed) do
def unfollow(%User{} = follower, %User{} = followed) do
ap_followers = followed.follower_address
if following?(follower, followed) do
if following?(follower, followed) and follower.ap_id != followed.ap_id do
following = follower.following
|> List.delete(ap_followers)
{ :ok, follower } = follower
|> follow_changeset(%{following: following})
|> Repo.update
|> update_and_set_cache
{:ok, followed} = update_follower_count(followed)
......@@ -172,6 +191,17 @@ def get_by_ap_id(ap_id) do
Repo.get_by(User, ap_id: ap_id)
end