diff options
Diffstat (limited to 'python/compare-locales/docs')
| -rw-r--r-- | python/compare-locales/docs/glossary.rst | 26 | ||||
| -rw-r--r-- | python/compare-locales/docs/index.rst | 191 |
2 files changed, 217 insertions, 0 deletions
diff --git a/python/compare-locales/docs/glossary.rst b/python/compare-locales/docs/glossary.rst new file mode 100644 index 000000000..e89839b16 --- /dev/null +++ b/python/compare-locales/docs/glossary.rst @@ -0,0 +1,26 @@ +======== +Glossary +======== + +.. glossary:: + :sorted: + + Localization + The process of creating content in a native language, including + translation, but also customizations like Search. + + Localizability + Enabling a piece of software to be localized. This is mostly + externalizing English strings, and writing build support to + pick up localized search engines etc. + + L10n + *Numeronym* for Localization, *L*, 10 chars, *n* + + L12y + Numeronym for Localizability + + l10n-merge + nick-name for the process of merging ``en-US`` and a particular + localization into one joint artifact without any missing strings, and + without technical errors, as far as possible. diff --git a/python/compare-locales/docs/index.rst b/python/compare-locales/docs/index.rst new file mode 100644 index 000000000..925ca0f88 --- /dev/null +++ b/python/compare-locales/docs/index.rst @@ -0,0 +1,191 @@ +============ +Localization +============ + +.. toctree:: + :maxdepth: 1 + + glossary + +The documentation here is targeted at developers, writing localizable code +for Firefox and Firefox for Android, as well as Thunderbird and SeaMonkey. + +If you haven't dealt with localization in gecko code before, it's a good +idea to check the :doc:`./glossary` for what localization is, and which terms +we use for what. + +Exposing strings +---------------- + +Localizers only handle a few file formats in well-known locations in the +source tree. + +The locations are in directories like + + :file:`browser/`\ ``locales/en-US/``\ :file:`subdir/file.ext` + +The first thing to note is that only files beneath :file:`locales/en-US` are +exposed to localizers. The second thing to note is that only a few directories +are exposed. Which directories are exposed is defined in files called +``l10n.ini``, which are at a +`few places <https://dxr.mozilla.org/mozilla-central/search?q=path%3Al10n.ini&redirect=true>`_ +in the source code. + +An example looks like this + +.. code-block:: ini + + [general] + depth = ../.. + + [compare] + dirs = browser + browser/branding/official + + [includes] + toolkit = toolkit/locales/l10n.ini + +This tells the l10n infrastructure three things: Resolve the paths against the +directory two levels up, include files in :file:`browser/locales/en-US` and +:file:`browser/branding/official/locales/en-US`, and load more data from +:file:`toolkit/locales/l10n.ini`. + +For projects like Thunderbird and SeaMonkey in ``comm-central``, additional +data needs to be provided when including an ``l10n.ini`` from a different +repository: + +.. code-block:: ini + + [include_toolkit] + type = hg + mozilla = mozilla-central + repo = http://hg.mozilla.org/ + l10n.ini = toolkit/locales/l10n.ini + +This tells the l10n pieces where to find the repository, and where inside +that repository the ``l10n.ini`` file is. This is needed because for local +builds, :file:`mail/locales/l10n.ini` references +:file:`mozilla/toolkit/locales/l10n.ini`, which is where the comm-central +build setup expects toolkit to be. + +Now that the directories exposed to l10n are known, we can talk about the +supported file formats. + +File formats +------------ + +This is just a quick overview, please check the +`XUL Tutorial <https://developer.mozilla.org/docs/Mozilla/Tech/XUL/Tutorial/Localization>`_ +for an in-depth tour. + +The following file formats are known to the l10n tool chains: + +DTD + Used in XUL and XHTML. Also for Android native strings. +Properties + Used from JavaScript and C++. When used from js, also comes with + `plural support <https://developer.mozilla.org/docs/Mozilla/Localization/Localization_and_Plurals>`_. +ini + Used by the crashreporter and updater, avoid if possible. +foo.defines + Used during builds, for example to create file:`install.rdf` for + language packs. + +Adding new formats involves changing various different tools, and is strongly +discouraged. + +Exceptions +---------- +Generally, anything that exists in ``en-US`` needs a one-to-one mapping in +all localizations. There are a few cases where that's not wanted, notably +around search settings and spell-checking dictionaries. + +To enable tools to adjust to those exceptions, there's a python-coded +:py:mod:`filter.py`, implementing :py:func:`test`, with the following +signature + +.. code-block:: python + + def test(mod, path, entity = None): + if does_not_matter: + return "ignore" + if show_but_do_not_merge: + return "report" + # default behavior, localizer or build need to do something + return "error" + +For any missing file, this function is called with ``mod`` being +the *module*, and ``path`` being the relative path inside +:file:`locales/en-US`. The module is the top-level dir as referenced in +:file:`l10n.ini`. + +For missing strings, the :py:data:`entity` parameter is the key of the string +in the en-US file. + +l10n-merge +---------- + +Gecko doesn't support fallback from a localization to ``en-US`` at runtime. +Thus, the build needs to ensure that the localization as it's built into +the package has all required strings, and that the strings don't contain +errors. To ensure that, we're *merging* the localization and ``en-US`` +at build time, nick-named :term:`l10n-merge`. + +The process is usually triggered via + +.. code-block:: bash + + $obj-dir/browser/locales> make merge-de LOCALE_MERGEDIR=$PWD/merge-de + +It creates another directory in the object dir, :file:`merge-ab-CD`, in +which the modified files are stored. The actual repackaging process looks for +the localized files in the merge dir first, then the localized file, and then +in ``en-US``. Thus, for the ``de`` localization of +:file:`browser/locales/en-US/chrome/browser/browser.dtd`, it checks + +1. :file:`$objdir/browser/locales/merge-de/browser/chrome/browser/browser.dtd` +2. :file:`$(LOCALE_BASEDIR)/de/browser/chrome/browser/browser.dtd` +3. :file:`browser/locales/en-US/chrome/browser/browser.dtd` + +and will include the first of those files it finds. + +l10n-merge modifies a file if it supports the particular file type, and there +are missing strings which are not filtered out, or if an existing string +shows an error. See the Checks section below for details. + +Checks +------ + +As part of the build and other localization tool chains, we run a variety +of source-based checks. Think of them as linters. + +The suite of checks is usually determined by file type, i.e., there's a +suite of checks for DTD files and one for properties files, etc. An exception +are Android-specific checks. + +Android +^^^^^^^ + +For Android, we need to localize :file:`strings.xml`. We're doing so via DTD +files, which is mostly OK. But the strings inside the XML file have to +satisfy additional constraints about quotes etc, that are not part of XML. +There's probably some historic background on why things are the way they are. + +The Android-specific checks are enabled for DTD files that are in +:file:`mobile/android/base/locales/en-US/`. + +Localizations +------------- + +Now that we talked in-depth about how to expose content to localizers, +where are the localizations? + +We host a mercurial repository per locale and per branch. Most of our +localizations only work starting with aurora, so the bulk of the localizations +is found on https://hg.mozilla.org/releases/l10n/mozilla-aurora/. We have +several localizations continuously working with mozilla-central, those +repositories are on https://hg.mozilla.org/l10n-central/. + +You can search inside our localized files on +`Transvision <https://transvision.mozfr.org/>`_ and +http://dxr.mozilla.org/l10n-mozilla-aurora/. |