Framework for live migrations
One of our larger problems is migrating our data out of jsonb into proper relations in Postgres because the migrations are so expensive. Could we solve this with live migrations?
e.g., attachments
- add code that can check if the target migration is complete, and if not complete yet access old functions that fetch data the current way
- design a migration framework that can resume if interrupted. Maybe make these migrations use a new Repo with 2 dedicated database connections.
- Run the migration in the background nonstop or in batches
- Users experience higher load while the migration is running, but doesn't require their instance stay down.
- This migration could also run offline, perhaps have a setting that skips it during
mix ecto.migrate
if you want to run all possible migrations online.