What is Five?

Five is a Zope 2 product that allows you to integrate Zope 3 technologies into Zope 2, today. It allows you to use all parts of the Zope 3 component architecture within Zope 2, namely interfaces, adapters and views. It also re-implements a few features known from Zope 3, such as the automatic generation of add and edit forms based on schemas, so that they can be used in a Zope 2 environment.

About the author

Philipp von Weitershausen attends Dresden University of Technology for Physics. He is also an independent software developer, a main contributor to Zope 3 and Five and the author of the Springer-Verlag book Web Component Development with Zope 3.

Target audience

This tutorial targets Zope 2 developers who in some way or another have to deal with i18n. That means the future of i18n in Zope 2 should concern them, so this is what this document is about. The reader should be familiar with i18n-aware development and familiar with current Zope-2-based solutions.

Localizer

There are a few third party Zope 2 products who make use of the translation service hook to register themselves as a translation service. The ones still used seem to be PlacelessTranslationService, an implementation modeled after a pre-alpha Zope 3 translation service, and Localizer, a Zope 2 product that has been around for a long time and is one of the pioneer works of Zope i18n.

Though plugging into a global API, Localizer places translations locally within the ZODB in MessageCatalog objects. The user may upload and download standard gettext message catalogs and/or simply edit translations via the ZMI. Localizer also automatically picks up new translation strings that it doesn't know about yet.

One of the biggest users of Localizer is CPS; it uses its own fork of Localizer while the main version is being maintained by the original author.

PlacelessTranslationService (PTS)

PlacelessTranslationService on the other hand is, as its name suggests, placeless. A more modern word for "placeless" is "global". PTS was initially a port of Zope 3's global translation service (not existent anymore) and has been heavily modified since, mostly to cope with Zope 2 scenarios. It reads gettext message catalogs from the filesystem by automatically looking through product directories looking for either

• an i18n directory containing message catalogs named like <domain>-<lang>.po (e.g. plone-de.po), or

• a locales directory with the standard gettext directory layout, e.g.:

  locales/
  locales/de/
  locales/de/LC_MESSAGES/
  locales/de/LC_MESSAGES/plone.po
  locales/de/LC_MESSAGES/archetypes.po
  locales/es/LC_MESSAGES/plone.po
  locales/es/LC_MESSAGES/archetypes.po
  ...
  

When PTS encounters .po files without their compiled .mo form, it automatically compiles them.

PTS is used by Plone and Silva and is often seen as a big source of trouble, especially when it comes to unicode issues. Like Localizer it also supports catching messages that it doesn't know about yet.

Both translation services provide a pluggable way of language extraction and negotiation, defaulting to the Accept-Language HTTP header and some proprietary settings. They also seem to mix the two concepts of (extraction vs. negotiation) and don't allow a clear distinction.

The i18n architecture in Zope 3

The Zope 3 i18n system consists of three major components:

• Translation data is provided by translation domain objects. These are registered as utilities providing ITranslationDomain and are looked up by their domain name, e.g.:

  >>> five_domain = zapi.getUtility(ITranslationDomain, 'five')
  

  The fact that the translations of different i18n domains are stored in different objects means that they don't all have to be from the same source. For example, the default translation domain implementation reads data from gettext .mo message catalogs, but you could implement your own version that reads translation data from XML files, for example. There also is a persistent translation domain implementation that stores translations in the ZODB; in this case the translation domain object is registered as a local utility. As you can see, you can decide how to provide translation data on a per-domain basis.

• The language extractor is responsible for extracting the user's preferred language from the request. The extractor is implemented as a IUserPreferredLanguages adapter for IHTTPRequest. That means if you want to find out which languages the user prefers, you simply have to do something like this:

  >>> preferred_lang = IUserPreferredLanguages(request)
  >>> preferred_lang.getPreferredLanguages()
  ['de', 'en']
  

  The default language extractor implementation extracts the information from the Accept-Language HTTP header. Other implementations like getting the language from a form variable or a cookie setting would be quite easy to implement (it's in fact shown in my book).

• The language negotiator takes the list of preferred languages and compares it with the list of available languages. It will then try to find the best match. This component is implemented as a utility providing INegotiator. Unlike the IUserPreferredLanguages adapter, it seldomly will have to be overridden, unless one wanted to customize the actual negotiation process.

In typical Zope 3 applications (which includes Five-based packages), user messages (in other words, unicode strings containing text that's intended to be presented to the user) can occur in three places:

• Page Templates (usage identical to the Zope 2 scenario),
• Python code (e.g. in a schema definition whose field titles and descriptions appear in an automatically generated form),
• ZCML (e.g. labels of menu items)

An extraction utility shipping with Zope 3 can extract translatable messages out of all three.

Messages in Python code

In ZPT, translatable text is marked with TAL commands from the i18n namespace. In ZCML, the ZCML machinery itself knows which directive parameters are supposed to be translated and treats the user input accordingly. In Python code, however, one has to explicitly mark text as translatable, for two reasons:

• The i18n extraction utility needs to know which strings should be extracted so that it can collect them.
• Just a unicode string is not enough information for the translation machinery. A translatable message needs to have at least some translation domain info, and possibly an explicit message ID and a default translation. It gets even more complicated when variable interpolation is involved.

To turn unicode strings into translatable messages, we use a message factory that creates i18n messages of a certain domain:

>>> from zope.i18nmessageid import MessageIDFactory
>>> _ = MessageIDFactory('five')

>>> msg = _(u'Translate this!')
>>> msg
u'Translate this'
>>> msg.domain
'five'

Note that with Zope 2.9+/3.2+, you should use MessageFactory instead of MessageIDFactory. For more information on that subject as well as some i18n message examples, please see the Turning MessageIDs into Rocks proposal as well as the relevant chapters in my book.

By the way, the reason the message factory is simply named _ is pure convention: the message extraction utitility will look for this identifier when collecting message IDs.

Also note that turning a unicode string into a translatable message isn't same as translating the string in-place. We're only marking them as translatable so that some other code will later do the translation for us. In typical Zope applications, that is some ZPT template because user messages sooner or later end up in HTML. In some rare cases you still might want to translate i18n messages manually; you can do that by invoking the zope.i18n.translate function:

>>> import zope.i18n
>>> zope.i18n.translate(msg, context=request)
u'Traduzca esto!'

Message catalogs

As said above, by default Zope 3 supports gettext message catalogs (in the compiled .mo form) arranged in the standard gettext directory format. Such a directory will have to be registered explicitly through ZCML, though. By convention, it is named locales, hence the ZCML would look something like this:

<configure
    xmlns="http://namespaces.zope.org/zope"
    xmlns:i18n="http://namespaces.zope.org/i18n"
    >
  <i18n:registerTranslations directory="locales" />
</configure>

This will scan the locales directory and register translation domain utilities according to .mo files found.

If you are not familiar with ZCML yet, I suggest reading over Step 2 of my "Five for Plone developers tutorial".

Backward compatibility

For now Zope 2 still supports the old-style translation services like PTS and Localizer. However, when Five 1.1 or greater is installed (and it is always in Zope 2.9 and higher), translation domains defined in Zope 3 take priority. So if you've registered some translation domains the Zope 3 way, their messages will always be translated through Five, never through PTS or Localizer.

Of course, for pages that contain messages from different domains, you would still expect consistent behaviour regarding the chosen language. When one message is translated the Zope 3 way because it's in a Zope 3 translation domain and the other is translated by an old-style translation service, you would really like both message to be translated into the same language. That means that both systems need to come to the same conclusion as to which language should be used.

Therefore, to ease the migration path for such cases where not all translation domains are handled by the Zope 3 system yet, Five provides language extractors that always come to the same conlusion as PTS or Localizer, respectively. How to set this up is documented in the Products/Five/doc/i18n.txt file which is also available on the Five website in HTML form.

There's more

Obviously, simply translating text from one language to another isn't all there's to internationalization. Another important part is localization which deals with the correct interpretation and representation of things like numbers, dates, times and such. The zope.i18n package provides a great deal of localization data and a rich API to deal with things like number and date formatting. Again, I have to refer you to the relevant chapters of my book.

Further reading

If you've enjoyed this tutorial and like my style of writing as well as the technology involved, please consider buying my book Web Component Development with Zope 3. The income from the book will allow me to continue to contribute to open source projects like Zope 3 and Five as well as create further documentation like this tutorial. Of course, you can also hire me.

Other resources:

• Zope 3 Appetizers, a list of Zope 3-related online tutorials
• Online version of Stephan Richter's Zope 3 Developers Book (a bit outdated and not in-sync with the printed version of the book)
• The Five mailinglist at z3-five@codespeak.net
• The IRC channel #z3-base on the Freenode network (irc.freenode.net) where Five developers hang out.