Backend translations

tusooa commented

2022-02-21 01:43:29 +00:00

Owner

Problem: backend-rendered pages are currently not translated. (For example:
https://git.pleroma.social/pleroma/pleroma/-/blob/develop/lib/pleroma/web/templates/twitter_api/remote_follow/follow.html.eex
https://git.pleroma.social/pleroma/pleroma/-/blob/develop/lib/pleroma/web/templates/o_auth/mfa/totp.html.eex
)

First, I think the current implementation in https://git.pleroma.social/pleroma/pleroma/-/blob/develop/lib/pleroma/web/gettext.ex is wrong, as Gettext is only aware of the locale set on the server machine, not what is requested by the user. set_locale_plug.ex seems to do the job.

If we stick to gettext, we should use with_locale/3 and analyse the language from the http request.

Frontend should set cookies whenever the user manually chooses a language. Then backend can read that cookie to detect language. Frontend and backend should use the same set of language codes.

Locales are expressed as strings (like "en" or "fr"); they can be arbitrary strings as long as they match a directory name.
If there is no corresponding cookie, read from the Accept-Language header, which should be the counterpart of navigator.languages.
If there is no such header, fallback to the default locale set in the config file.
If there is no such config entry, fallback to library default ("en").

Something else to consider:

with_locale/3:

This function just sets the Gettext locale for backend to locale before running fun and sets it back to its previous value afterwards.

Any concurrency issues with this?
As far as I know, gettext does not have a fallback sequence for desired languages. However, both navigator.languages and Accept-Language allow choosing multiple languages. This problem arises because, for example, a user who chooses "Simplified Chinese" (zh-Hans) as their first preferred language may prefer "Traditional Chinese" (zh-Hant) over "English" (en). But in gettext if there is no translation in zh-Hans it will just fallback to the untranslated msgid.

As an alternative to gettext, I think fluent is a more satisfying solution, as it (at least the js implementation) solves the sequence problem.

There are a couple of elixir implementations in the wild:
- https://github.com/Virviil/libfluent (still does not support language sequences... and it seems to use rust)
- https://hex.pm/packages/fluent (unmaintained for years)
gettext uses untranslated string as the identifiers. fluent and vue-i18n uses opaque codes as identifiers. Benefits of the former may be ease of testing (but this can be bypassed by not testing rendered strings). Benefits of the latter is you get finer control of what needs to be re-translated (if you fix a typo in the reference string, it apparently does not need retranslation, whereas if you use some text for different purposes, it may need retranslation even if the reference text is the same.)
fluent (syntax) is also able to resolves some numerical cases. For example, if there are multiple numbers in a sentence (e.g. "1 out of 3"), the translated text may need to agree with both numbers. gettext and vue-i18n cannot do this without separating it into multiple entries.

Problem: backend-rendered pages are currently not translated. (For example: https://git.pleroma.social/pleroma/pleroma/-/blob/develop/lib/pleroma/web/templates/twitter_api/remote_follow/follow.html.eex https://git.pleroma.social/pleroma/pleroma/-/blob/develop/lib/pleroma/web/templates/o_auth/mfa/totp.html.eex ) ~~First, I think the current implementation in https://git.pleroma.social/pleroma/pleroma/-/blob/develop/lib/pleroma/web/gettext.ex is wrong, as Gettext is only aware of the locale set on the server machine, not what is requested by the user.~~ set_locale_plug.ex seems to do the job. If we stick to gettext, we should use [with_locale/3](https://hexdocs.pm/gettext/Gettext.html#with_locale/3) and analyse the language from the http request. - Frontend should set cookies whenever the user manually chooses a language. Then backend can read that cookie to detect language. Frontend and backend should use the same set of language codes. > Locales are expressed as strings (like "en" or "fr"); they can be arbitrary strings as long as they match a directory name. - If there is no corresponding cookie, read from the Accept-Language header, which should be the counterpart of `navigator.languages`. - If there is no such header, fallback to the default locale set in the config file. - If there is no such config entry, fallback to library default ("en"). Something else to consider: - with_locale/3: > This function just sets the Gettext locale for backend to locale before running fun and sets it back to its previous value afterwards. Any concurrency issues with this? - As far as I know, gettext does not have a fallback sequence for desired languages. However, both `navigator.languages` and `Accept-Language` allow choosing multiple languages. This problem arises because, for example, a user who chooses "Simplified Chinese" (zh-Hans) as their first preferred language may prefer "Traditional Chinese" (zh-Hant) over "English" (en). But in gettext if there is no translation in zh-Hans it will just fallback to the untranslated `msgid`. As an alternative to gettext, I think [fluent](https://github.com/projectfluent/fluent) is a more satisfying solution, as it (at least the js implementation) solves the sequence problem. There are a couple of elixir implementations in the wild: - https://github.com/Virviil/libfluent (still does not support language sequences... and it seems to use rust) - https://hex.pm/packages/fluent (unmaintained for years) - gettext uses *untranslated string* as the identifiers. fluent and vue-i18n uses opaque codes as identifiers. Benefits of the former may be ease of testing (but this can be bypassed by not testing rendered strings). Benefits of the latter is you get finer control of what needs to be re-translated (if you fix a typo in the reference string, it apparently does not need retranslation, whereas if you use some text for different purposes, it may need retranslation even if the reference text is the same.) - fluent (syntax) is also able to resolves some numerical cases. For example, if there are multiple numbers in a sentence (e.g. "1 out of 3"), the translated text may need to agree with both numbers. gettext and vue-i18n cannot do this without separating it into multiple entries.

tusooa commented

2022-02-21 23:48:10 +00:00

Author

Owner

Also need to set up weblate. Currently there is no backend translations (were they lost?) at https://translate.pleroma.social/projects/pleroma/ .

tusooa commented

2022-02-26 16:15:33 +00:00

Author

Owner

Problem: Email is also backend-rendered, but I guess there would be no corresponding Request here.

tusooa commented

2022-02-27 19:40:57 +00:00

Author

Owner

can make user choose a language when signing up, stored/changeable in user settings

Rows
Columns

Backend translations #60