No federation. 2 separate pleroma and nginx machines. Pleroma reports: request_id= [error] Could not decode user at fetch https://123.xyz, :econnrefused
Hello everyone.
I have installed the pleroma as I am trying to self-host it.
My use case is atypical to say the least and long story short - my pleroma instance fails to federate.
Long story:
I have 2 machines:
-
MacMini 3.2 (physical machine) running Debian 11 IP 192.168.1.5 - This is where Pleroma lives on Port 4000
-
Second machine is slightly more complicated. Its a Virtual Machine running on Proxmox server. Proxmox server IP 192.168.1.12 has a bunch of Virtual Machines running. The Virtual Machine that we are interested in is running Arch Linux, its configured to serve https on port 30303 from the IP 192.168.2.50 using nginx / lets encrypt certs. IPTABLES are configured to forward all traffic from 192.168.2.50 to 192.168.1.12 on port 30303 and vice versa.
Yes its a weird setup.
Yes it works.
Why port 30303? Because my ISP's router does not allow port 443 forwarding. I had to make due. It's working and can be tested here:
https://andrzejl.eu:30303 page is not impressive - its a placeholder / leftover after I nuked my wordpress installation - but it works.
##########
How is my nginx configured?
[root@andrzejl andrzejl]# grep -v '#' /etc/nginx/sites_enabled/fediverse.andrzejl.eu
proxy_cache_path /tmp/pleroma-media-cache levels=1:2 keys_zone=pleroma_media_cache:10m max_size=10g
inactive=720m use_temp_path=off;
upstream phoenix {
server 192.168.1.5:4000 max_fails=5 fail_timeout=60s;}
server {
server_name fediverse.andrzejl.eu;
listen 10101;
location / {
return 301 https://$server_name$request_uri;
}
}
ssl_session_cache shared:ssl_session_cache:10m;
server {
server_name fediverse.andrzejl.eu;
listen 30303 ssl http2;
ssl_session_timeout 1d;
ssl_session_cache shared:MozSSL:10m;
ssl_session_tickets off;
ssl_trusted_certificate /etc/letsencrypt/live/andrzejl.eu-0001/chain.pem;
ssl_certificate /etc/letsencrypt/live/andrzejl.eu-0001/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/andrzejl.eu-0001/privkey.pem;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers "ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4";
ssl_prefer_server_ciphers off;
ssl_ecdh_curve X25519:prime256v1:secp384r1:secp521r1;
ssl_stapling on;
ssl_stapling_verify on;
gzip_vary on;
gzip_proxied any;
gzip_comp_level 6;
gzip_buffers 16 8k;
gzip_http_version 1.1;
gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript application/activity+json application/atom+xml;
client_max_body_size 16m;
ignore_invalid_headers off;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
location / {
proxy_pass http://phoenix;
}
location ~ ^/(media|proxy) {
proxy_cache pleroma_media_cache;
slice 1m;
proxy_cache_key $host$uri$is_args$args$slice_range;
proxy_set_header Range $slice_range;
proxy_cache_valid 200 206 301 304 1h;
proxy_cache_lock on;
proxy_ignore_client_abort on;
proxy_buffering on;
chunked_transfer_encoding on;
proxy_pass http://phoenix;
}
location /proxy {
proxy_cache pleroma_media_cache;
proxy_cache_lock on;
proxy_pass http://192.168.1.5:4000;
}
access_log /var/log/nginx/fediverse.andrzejl.eu_ssl_access.log;
error_log /var/log/nginx/fediverse.andrzejl.eu_ssl_error.log;
}
[root@andrzejl andrzejl]#
##########
The config works. Pleroma installation can be accessed via:
https://fediverse.andrzejl.eu:30303/
IF it shows error 502 when you try to access it - I am probably recompiling pleroma or testing something.
##########
How was pleroma installed?
After fully updating Debian 11 I ran this command to install all the necessary apps and their deps:
sudo apt install git build-essential postgresql postgresql-contrib postgresql-13-rum cmake libmagic-dev elixir erlang-dev erlang-nox imagemagick ffmpeg libimage-exiftool-perl
And then I ran this set of commands:
sudo useradd -r -s /bin/false -m -d /var/lib/pleroma -U pleroma
sudo mkdir -p /opt/pleroma
sudo chown -R pleroma:pleroma /opt/pleroma
sudo mkdir -p /var/lib/pleroma
sudo chown -R pleroma:pleroma /var/lib/pleroma
sudo -Hu pleroma git clone -b stable https://git.pleroma.social/pleroma/pleroma /opt/pleroma
cd /opt/pleroma
sudo -Hu pleroma MIX_ENV=prod mix deps.get
########## Chose y to install Hex
sudo -Hu pleroma MIX_ENV=prod mix pleroma.instance gen
########## Chose y to install rebar3
########## IN MY CASE I EDIT port from 443 to 30303 and enable media proxy in the /opt/pleroma/config/generated_config.exs
mcedit /opt/pleroma/config/generated_config.exs
########## # This is the port line
########## url: [host: "fediverse.andrzejl.eu", scheme: "https", port: 30303],
########## # This is the media proxy enabling bit
########## config :pleroma, :media_proxy,
########## enabled: true,
########## proxy_opts: [
########## redirect_on_failure: true
########## ]
########## # base_url: "https://cache.pleroma.social"
sudo -Hu pleroma cp config/{generated_config.exs,prod.secret.exs}
sudo -Hu postgres psql -f config/setup_db.psql
sudo -Hu pleroma MIX_ENV=prod mix ecto.migrate
sudo -Hu pleroma MIX_ENV=prod mix ecto.migrate --migrations-path priv/repo/optional_migrations/rum_indexing/
sudo cp /opt/pleroma/installation/pleroma.service /etc/systemd/system/pleroma.service
sudo systemctl enable --now pleroma.service
########## This will start the pleroma instance
sudo -Hu pleroma MIX_ENV=prod mix phx.server
sudo -Hu pleroma MIX_ENV=prod mix pleroma.user new USERNAME USERNAME@SOME.EMAIL --admin --moderator
Then I got the password resetting link and I was able to change the password for my admin / mod user and log into the pleroma.
########## Disable registration of new users and add 4 lines as the last 4 lines in in /opt/pleroma/config/prod.secret.exs
mcedit /opt/pleroma/config/prod.secret.exs
########## # Disable registrations
########## registrations_open: false
########## # Those 4 lines are added after many hours of research BUT I am not sure they are even needed.
########## # The last 2 lines seem to cause problems - a friend who follows me is not getting @ notifications
########## # However adding them triggers something and after a while pleroma tries to federate with error but it tries
########## config :pleroma, Pleroma.Plugs.RemoteIP, enabled: false
########## config :pleroma, :rate_limit, nil
########## config :pleroma, :http,
########## proxy_url: "127.0.0.1:8123"
su -
reboot
#############
So now to the juicy bits. What does my pleroma config looks like? I sure hope I removed all the sensitive bits...
root@MacMiniServer:/home/andrzejl# grep -v '#' /opt/pleroma/config/prod.secret.exs
import Config
config :pleroma, Pleroma.Web.Endpoint,
url: [host: "fediverse.andrzejl.eu", scheme: "https", port: 30303],
http: [ip: {192, 168, 1, 5}, port: 4000],
secret_key_base: "SECRETKEY",
signing_salt: "SIGNINGSALT"
config :pleroma, :instance,
name: "Git clone pleroma...",
email: "USERNAME@SOME.EMAIL",
notify_email: "USERNAME@SOME.EMAIL",
limit: 5000,
registrations_open: false
config :pleroma, :media_proxy,
enabled: true,
proxy_opts: [
redirect_on_failure: true
]
config :pleroma, Pleroma.Repo,
adapter: Ecto.Adapters.Postgres,
username: "pleroma",
password: "SUPERMEGAINCREDIBLYSTRONGPASSWORDOHBOY!",
database: "pleroma",
hostname: "localhost"
config :web_push_encryption, :vapid_details,
subject: "mailto:USERNAME@SOME.EMAIL",
public_key: "PUBLICKEY",
private_key: "PRIVATEKEY"
config :pleroma, :database, rum_enabled: true
config :pleroma, :instance, static_dir: "/opt/pleroma/instance/static"
config :pleroma, Pleroma.Uploaders.Local, uploads: "/opt/pleroma/uploads"
config :joken, default_signer: "DEFAULT_SIGNER"
config :pleroma, configurable_from_database: true
config :pleroma, Pleroma.Upload, filters: [Pleroma.Upload.Filter.Exiftool]
config :pleroma, Pleroma.Plugs.RemoteIP, enabled: false
config :pleroma, :rate_limit, nil
config :pleroma, :http,
proxy_url: "127.0.0.1:8123"
root@MacMiniServer:/home/andrzejl#
####################
Upon reboot the instance is starting beautifully however after a while I would get several messages:
journalctl -u pleroma --no-pager -b -f | grep error
Feb 08 20:33:20 MacMiniServer mix[1233]: 20:33:20.998 request_id=FtHpy-pITpyvdTEAAUzx [error] Could not decode user at fetch https://toot.wales/users/ChrisWere, :econnrefused
Feb 08 20:33:21 MacMiniServer mix[1233]: 20:33:21.041 request_id=FtHpy-zWEH-1QvYAAU0B [error] Could not decode user at fetch https://nrw.social/users/schipplock, :econnrefused
Feb 08 20:33:21 MacMiniServer mix[1233]: 20:33:21.111 request_id=FtHpy_EEgRhPJQ0AAU0R [error] Could not decode user at fetch https://mastodon.social/users/fribbledom, :econnrefused
Feb 08 20:33:21 MacMiniServer mix[1233]: 20:33:21.207 request_id=FtHpy_a6iLwiqIcAAU0h [error] Could not decode user at fetch https://toot.wales/actor, :econnrefused
Feb 08 20:33:21 MacMiniServer mix[1233]: 20:33:21.259 request_id=FtHpy_nOGrBOPvcAAU0x [error] Could not decode user at fetch https://nrw.social/actor, :econnrefused
Feb 08 20:33:21 MacMiniServer mix[1233]: 20:33:21.296 request_id=FtHpy_wM7fxvTn0AAU1B [error] Could not decode user at fetch https://mastodon.technology/users/travisfw, :econnrefused
Feb 08 20:33:21 MacMiniServer mix[1233]: 20:33:21.411 request_id=FtHpzALosRvawj8AAU1R [error] Could not decode user at fetch https://mastodon.social/actor, :econnrefused
Feb 08 20:33:21 MacMiniServer mix[1233]: 20:33:21.681 request_id=FtHpzBMAnGK8T70AAU1h [error] Could not decode user at fetch https://linuxrocks.online/users/ChrisWere, :econnrefused
Feb 08 20:33:21 MacMiniServer mix[1233]: 20:33:21.740 request_id=FtHpzBaEM4xoQUoAAU1x [error] Could not decode user at fetch https://mastodon.technology/actor, :econnrefused
Feb 08 20:33:22 MacMiniServer mix[1233]: 20:33:22.537 request_id=FtHpzEX_nIYObacAAU2B [error] Could not decode user at fetch https://linuxrocks.online/actor, :econnrefused
Feb 08 20:34:34 MacMiniServer mix[1233]: 20:34:34.982 request_id=FtHp3SQ4N9z9mcEAATeS [error] Could not decode user at fetch https://toot.wales/actor, :econnrefused
Feb 08 20:34:35 MacMiniServer mix[1233]: 20:34:35.021 request_id=FtHp3SaN6Xx8eBwAAU_h [error] Could not decode user at fetch https://nrw.social/actor, :econnrefused
Feb 08 20:34:35 MacMiniServer mix[1233]: 20:34:35.312 request_id=FtHp3Te0QwLZ9QEAATei [error] Could not decode user at fetch https://fosstodon.org/actor, :econnrefused
Feb 08 20:34:35 MacMiniServer mix[1233]: 20:34:35.405 request_id=FtHp3T1ChWiUONsAAU_x [error] Could not decode user at fetch https://mastodon.technology/actor, :econnrefused
Feb 08 20:34:35 MacMiniServer mix[1233]: 20:34:35.714 request_id=FtHp3U-rzdlk8eYAAVAB [error] Could not decode user at fetch https://mastodon.social/actor, :econnrefused
Feb 08 20:34:36 MacMiniServer mix[1233]: 20:34:36.014 request_id=FtHp3WGSYNeuwDIAAVAR [error] Could not decode user at fetch https://linuxrocks.online/actor, :econnrefused
And the federation fails.
Now I do hope I explained everything in a way that will not make your head spin but IF I didn't and you need clarifications I will do my best to reply to your questions / suggestions in a timely fashion but just in case I don't please be patient I will try and do so asahp.
Please let me know if you need more information like logs etc.
What am I missing / doing wrong?
Kindest regards.
Andrzej