Reapply "Custom mix task to retry failed tests once in CI pipeline"

There are some random test failures of which the root cause has not yet been identified. We'll bring this back and in the future just disable it instead of removing it entirely.