Trans is an Elixir library for managing translations embedded into data structures. While Trans can be used for fetching translations from maps or structs, it shines when coupled with Ecto for fetching and querying translations from the database.

The traditional approach to translation management consists on using database tables dedicated only to translation storage. For example we may have a posts table and a companion post_translations table. This is the approach used by the Ruby gem Globalize.

While this approach is battle tested and works, it has a few disadvantages that can be improved:

  • It complicates the database schema requiring a new table to be created for each translatable schema.
  • It complicates the database migrations, since changes in the structure of a table must be replicated into the translation tables.
  • It requires constant JOINs in order to filter or fetch records along with their translations. This has a visible impact on performance when multiple translatable schemas intervene in a query.

Modern RDBMSs support unstructured datatypes such as JSON. We can leverage this unstructured datatype support to embed the translations into a field of the schema itself, instead of having separated schemas for translations. This eliminates the need of maintaining duplicated database tables in sync and vastly reduces the number of JOIN clauses in queries.

While the database itself provides mechanisms for accessing unstructured data, Ecto itself does not provide a high level interface for accessing them easily. This is where Trans mission begins.

Ecto allows you to write custom SQL fragments that can be used, among other things, to access JSON fields in the database. Writing those SQL fragments for each query that requires translations can get tiresome and error prone quickly. Trans generates the SQL fragments for you and validates them during compile time.

Trans 1.0 sins

Bypassing Ecto syntax for queries

Ecto.Query provides a nice syntax for query generation. Queries can be written using a series of macros (or functions, depending on which API you prefer) that allow for parameter interpolation, data binding, composition and prefixing.

Instead of enhancing Ecto.Query.Api Trans provided its own, different, API that didn’t interact with it. This caused a series of inconveniences and limitations.

The first and most noticeable inconvenience appears when writting a query with some conditions on the translatable data. Instead of specifying the conditions when building the query, Trans 1.0 required to build the query first, and then pass it to a function that would add the required conditions.

The first and most noticeable inconvenience appears when writing a query with some conditions on the translatable data. With a normal Ecto query you wold specify the conditions when building the query itself. Instead, Trans 1.0 required to build the query first, and then pass it to a function that would add the required conditions.

# The QueryBuilder component bypassed Ecto.Query functionality.
# Ugly to say the least...
iex> Article
...> |> Trans.QueryBuilder.with_translation(:fr, :title, "La République")
...> |> Repo.all

Another problem of this approach is that conditions on translatable fields could be only added on the main schema of the query. Joined schemas and associations were out… Ouch!

Mixing responsibilities in schema modules

The translation container is the field that embeds the translations for the schema. Trans uses by default a field called translations, but allows this to be customized.

In Trans 1.0, you had two options: calling the QueryBuilder functions directly or using Trans in your schema module to get some convenience functions set up to avoid repetition. Setting up the convenience functions was the most comfortable option, but it added extra responsibilities and concerns to schema modules that did not belong there.

# Trans 1.0 polluted the modules interface and mixed concerns that should
# belong to other parts of the application. Some of the following functions
# have been automatically added by Trans.
iex(5)> Article.
changeset/1            changeset/2            with_translation/4
with_translation/5     with_translations/2    with_translations/3

Unsafe against translation errors

Trans 1.0 performed some basic checks against translation errors. For example, trying to add a query condition on an untranslatable field would result in a run-time ArgumentError.

Trans 2.0 improves those checks and produces errors in the compilation phase instead on run-time.

# Trans would let you add conditions on non-existing fields which resulted in run-time errors.
iex> Article
...> |> Article.with_translation(:es, :fake, "this field does not exist")
...> |> Repo.all
** (ArgumentError) The field `fake` is not declared as translatable

Trans 2.0 improvements

Embracing Ecto Syntax for queries

In Trans 2.0, the QueryBuilder component has been completely rewritten: it now runs in compilation time and it’s only mission is the generation of the required SQL fragment. All the different functions in the QueryBuilder module have been unified into the translated/3 macro.

Being a macro allows the QueryBuilder to generate the SQL fragment in compile time and add it to the query being compiled. This way we can use the translated/3 macro when creating Ecto queries and interact with the rest of functions and macros provided by Ecto.Query and Ecto.Query.Api.

# The translated/3 macro is compatible with Ecto.Query and Ecto.Query.Api

iex> Repo.all(from a in Article,
...>   join: c in assoc(a, :comments),
...>   where: translated(Article, a.title, :fr) == "Trans",
...>   where: ilike(translated(Comment, c.comment, :en), "good"))

Extracting responsabilities out of schema modules

With Trans 2.0, modules only export an underscore function named __trans__ that returns the Trans configuration for the module.

This keeps client modules clean and keeps the logic contained into the QueryBuilder and Translator modules instead of spreading it among the client code.

Safety checks as soon as possible

As explained before, Trans 1.0 did some basic checks in runtime. Since the translation configuration for each module is set up during the compilation, Trans 2.0 uses it to perform safety checks before runtime.

Want to translate a non existing field? Your application won’t compile. Want to translate a non-translatable field? Whooops… Your application won’t compile. Well… You get the idea. This makes Trans 2.0 safer and more approachable than the previous version.

Future plans

With this update Trans is in a good shape for being maintained and continuously improved. The main features that I want to include in future versions of Trans are:

  • Support for MySQL. MySQL 5.7 introduced a native JSON type that could be used by Trans in the same way as the PostgreSQL JSONB type. There is an open issue in the MySQL database driver repository to add support for this type.
  • Specify available locales and custom fallback sequences. Currently Trans falls back to the default language when a translation does not exist. This behavior could be customized to check other locales before falling back to the default language.
  • Make translations an embedded schema and create a custom Ecto type. This is still an unfinished idea, but the main objective here would be to represent translations as embedded schemas. This way the translations could specify their own changesets and perform required validations or transformations.

Wrapping up

Trans first release was on June 4th, 2016, so it will reach a year of life soon. This has been an interesting journey where we had 6 releases, 159 commits and have reached almost 250 downloads. Thank you all!

With this new release Trans has reached a level of maturity that makes it more usable, safe and maintainable for the time coming. So expect more improvements and features in the future.

If you found a bug or have any ideas or comments feel free to post there or to open an issue in GitHub. There are also interesting discussions in the Trans thread at Elixir Forum.

To finish this article I want to specially thank my friends Óscar de Arriba, Victor Ortiz, Enol Iglesias, Rubén Sierra and Iván González for all the support ❤️. See you at the ElixirConf.EU!