Cannot ecto.migrate to 2.1.0 with referential integrity issues
Environment
- Installation type (OTP or From Source): From Source
- Pleroma version (could be found in the "Version" tab of settings in Pleroma-FE): 2.1.0
- Elixir version (
elixir -v
for from source installations, N/A for OTP): N/A - Operating system: Linux 5.4.59-1-MANJARO-ARM #1 (closed) SMP PREEMPT
- PostgreSQL version (
psql -V
): 12.4
Bug description
While migrating to 2.1.0 I get this error:
Generated pleroma app
20:01:49.258 [info] == Running 20200602125218 Pleroma.Repo.Migrations.BackfillNotificationTypes.up/0 forward
** (FunctionClauseError) no function clause matching in Pleroma.MigrationHelper.NotificationBackfill.type_from_activity/1
The following arguments were given to Pleroma.MigrationHelper.NotificationBackfill.type_from_activity/1:
# 1
nil
Attempted function clauses (showing 1 out of 1):
defp type_from_activity(%{data: %{"type" => type}} = activity)
(pleroma 2.1.0) Pleroma.MigrationHelper.NotificationBackfill.type_from_activity/1
(pleroma 2.1.0) lib/pleroma/migration_helper/notification_backfill.ex:24: anonymous fn/1 in Pleroma.MigrationHelper.NotificationBackfill.fill_in_notification_types/0
(elixir 1.10.4) lib/enum.ex:789: anonymous fn/3 in Enum.each/2
(elixir 1.10.4) lib/enum.ex:3383: anonymous fn/3 in Enum.each/2
(elixir 1.10.4) lib/enum.ex:3686: Enumerable.List.reduce/3
(elixir 1.10.4) lib/stream.ex:1462: Stream.do_list_resource/6
(elixir 1.10.4) lib/enum.ex:3383: Enum.each/2
(ecto_sql 3.4.5) lib/ecto/migration/runner.ex:278: Ecto.Migration.Runner.perform_operation/3
That happened because when executing this:
[lib/pleroma/migration_helper/notification_backfill.ex:fill_in_notification_types]
19 query
20 |> Repo.chunk_stream(100)
21 |> Enum.each(fn notification ->
22 type =
23 notification.activity
24 |> type_from_activity()
25
26 notification
27 |> Ecto.Changeset.change(%{type: type})
28 |> Repo.update()
notification.activity
was NULL for some records in notifications
whose activity_id
did not match any corresponding record inside the table activities
.
I have cleaned up the notifications
table:
DELETE
FROM notifications
WHERE id IN (
SELECT notifications.id
FROM notifications
LEFT JOIN activities ON notifications.activity_id = activities.id
WHERE activities.id is null
);
and re-run mix ecto.migrate
and this time it went on.
I am filing this as a bug because:
- if referral integrity between the tables
notifications
andactivites
is needed, then the schema must be modified and an extra check in the migration procedure needs to be introduced - the instance was working fine until then, even with this inconsistency, therefore it looks like a regression.