<!DOCTYPE html>
<html lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="generator" content="Docutils 0.4: http://docutils.sourceforge.net/">
<title>Genshi: Internationalization and Localization</title>
<link rel="stylesheet" href="common/style/edgewall.css" type="text/css">
</head>
<body>
<div class="document" id="internationalization-and-localization">
<div id="navigation">
<span class="projinfo">Genshi 0.5.1</span>
<a href="index.html">Documentation Index</a>
</div>
<h1 class="title">Internationalization and Localization</h1>
<p>Genshi provides basic supporting infrastructure for internationalizing
and localizing templates. That includes functionality for extracting localizable
strings from templates, as well as a template filter that can apply translations
to templates as they get rendered.</p>
<p>This support is based on <a class="reference" href="http://www.gnu.org/software/gettext/">gettext</a> message catalogs and the <a class="reference" href="http://docs.python.org/lib/module-gettext.html">gettext Python
module</a>. The extraction process can be used from the API level, or through the
front-ends implemented by the <a class="reference" href="http://babel.edgewall.org/">Babel</a> project, for which Genshi provides a
plugin.</p>
<div class="contents topic">
<p class="topic-title first"><a id="contents" name="contents">Contents</a></p>
<ul class="auto-toc simple">
<li><a class="reference" href="#basics" id="id1" name="id1">1 Basics</a></li>
<li><a class="reference" href="#extraction" id="id2" name="id2">2 Extraction</a><ul class="auto-toc">
<li><a class="reference" href="#configuration-options" id="id3" name="id3">2.1 Configuration Options</a></li>
</ul>
</li>
<li><a class="reference" href="#translation" id="id4" name="id4">3 Translation</a></li>
<li><a class="reference" href="#related-considerations" id="id5" name="id5">4 Related Considerations</a><ul class="auto-toc">
<li><a class="reference" href="#unicode" id="id6" name="id6">4.1 Unicode</a></li>
<li><a class="reference" href="#date-and-time" id="id7" name="id7">4.2 Date and Time</a></li>
<li><a class="reference" href="#formatting-and-locale-data" id="id8" name="id8">4.3 Formatting and Locale Data</a></li>
</ul>
</li>
</ul>
</div>
<div class="section">
<h1><a id="basics" name="basics">1 Basics</a></h1>
<p>The simplest way to internationalize and translate templates would be to wrap
all localizable strings in a <tt class="docutils literal"><span class="pre">gettext()</span></tt> function call (which is often aliased
to <tt class="docutils literal"><span class="pre">_()</span></tt> for brevity). In that case, no extra template filter is required.</p>
<div class="highlight"><pre><span class="nt"><p></span><span class="cp">${</span><span class="n">_</span><span class="p">(</span><span class="s">"Hello, world!"</span><span class="p">)</span><span class="cp">}</span><span class="nt"></p></span>
</pre></div>
<p>However, this approach results in significant “character noise” in templates,
making them harder to read and preview.</p>
<p>The <tt class="docutils literal"><span class="pre">genshi.filters.Translator</span></tt> filter allows you to get rid of the
explicit <a class="reference" href="http://www.gnu.org/software/gettext/">gettext</a> function calls, so you can continue to just write:</p>
<div class="highlight"><pre><span class="nt"><p></span>Hello, world!<span class="nt"></p></span>
</pre></div>
<p>This text will still be extracted and translated as if you had wrapped it in a
<tt class="docutils literal"><span class="pre">_()</span></tt> call.</p>
<div class="note">
<p class="first admonition-title">Note</p>
<p class="last">For parameterized or pluralizable messages, you need to continue using
the appropriate <tt class="docutils literal"><span class="pre">gettext</span></tt> functions.</p>
</div>
<p>You can control which tags should be ignored by this process; for example, it
doesn't really make sense to translate the content of the HTML
<tt class="docutils literal"><span class="pre"><script></script></span></tt> element. Both <tt class="docutils literal"><span class="pre"><script></span></tt> and <tt class="docutils literal"><span class="pre"><style></span></tt> are excluded
by default.</p>
<p>Attribute values can also be automatically translated. The default is to
consider the attributes <tt class="docutils literal"><span class="pre">abbr</span></tt>, <tt class="docutils literal"><span class="pre">alt</span></tt>, <tt class="docutils literal"><span class="pre">label</span></tt>, <tt class="docutils literal"><span class="pre">prompt</span></tt>, <tt class="docutils literal"><span class="pre">standby</span></tt>,
<tt class="docutils literal"><span class="pre">summary</span></tt>, and <tt class="docutils literal"><span class="pre">title</span></tt>, which is a list that makes sense for HTML documents.
Of course, you can tell the translator to use a different set of attribute
names, or none at all.</p>
<p>In addition, you can control automatic translation in your templates using the
<tt class="docutils literal"><span class="pre">xml:lang</span></tt> attribute. If the value of that attribute is a literal string, the
contents and attributes of the element will be ignored:</p>
<div class="highlight"><pre><span class="nt"><p</span> <span class="na">xml:lang=</span><span class="s">"en"</span><span class="nt">></span>Hello, world!<span class="nt"></p></span>
</pre></div>
<p>On the other hand, if the value of the <tt class="docutils literal"><span class="pre">xml:lang</span></tt> attribute contains a Python
expression, the element contents and attributes are still considered for
automatic translation:</p>
<div class="highlight"><pre><span class="nt"><html</span> <span class="na">xml:lang=</span><span class="s">"</span><span class="nv">$locale</span><span class="s">"</span><span class="nt">></span>
...
<span class="nt"></html></span>
</pre></div>
</div>
<div class="section">
<h1><a id="extraction" name="extraction">2 Extraction</a></h1>
<p>The <tt class="docutils literal"><span class="pre">Translator</span></tt> class provides a class method called <tt class="docutils literal"><span class="pre">extract</span></tt>, which is
a generator yielding all localizable strings found in a template or markup
stream. This includes both literal strings in text nodes and attribute values,
as well as strings in <tt class="docutils literal"><span class="pre">gettext()</span></tt> calls in embedded Python code. See the API
documentation for details on how to use this method directly.</p>
<p>This functionality is integrated into the message extraction framework provided
by the <a class="reference" href="http://babel.edgewall.org/">Babel</a> project. Babel provides a command-line interface as well as
commands that can be used from <tt class="docutils literal"><span class="pre">setup.py</span></tt> scripts using <a class="reference" href="http://peak.telecommunity.com/DevCenter/setuptools">Setuptools</a> or
<a class="reference" href="http://docs.python.org/dist/dist.html">Distutils</a>.</p>
<p>The first thing you need to do to make Babel extract messages from Genshi
templates is to let Babel know which files are Genshi templates. This is done
using a “mapping configuration”, which can be stored in a configuration file,
or specified directly in your <tt class="docutils literal"><span class="pre">setup.py</span></tt>.</p>
<p>In a configuration file, the mapping may look like this:</p>
<div class="highlight"><pre><span class="c"># Python souce</span>
<span class="k">[python:**.py]</span>
<span class="c"># Genshi templates</span>
<span class="k">[genshi:**/templates/**.html]</span>
<span class="na">include_attrs</span> <span class="o">=</span> <span class="s">title</span>
<span class="k">[genshi:**/templates/**.txt]</span>
<span class="na">template_class</span> <span class="o">=</span> <span class="s">genshi.template.TextTemplate</span>
<span class="na">encoding</span> <span class="o">=</span> <span class="s">latin-1</span>
</pre></div>
<p>Please consult the Babel documentation for details on configuration.</p>
<p>If all goes well, running the extraction with Babel should create a POT file
containing the strings from your Genshi templates and your Python source files.</p>
<div class="note">
<p class="first admonition-title">Note</p>
<p class="last">Genshi currently does not support “translator comments”, i.e. text in
template comments that would get added to the POT file. This support
may or may not be added in future versions.</p>
</div>
<div class="section">
<h2><a id="configuration-options" name="configuration-options">2.1 Configuration Options</a></h2>
<p>The Genshi extraction plugin for Babel supports the following options:</p>
<div class="section">
<h3><a id="template-class" name="template-class">2.1.1 <tt class="docutils literal"><span class="pre">template_class</span></tt></a></h3>
<p>The concrete <tt class="docutils literal"><span class="pre">Template</span></tt> class that the file should be loaded with. Specify
the package/module name and the class name, separated by a colon.</p>
<p>The default is to use <tt class="docutils literal"><span class="pre">genshi.template:MarkupTemplate</span></tt>, and you'll want to
set it to <tt class="docutils literal"><span class="pre">genshi.template:TextTemplate</span></tt> for <a class="reference" href="text-templates.html">text templates</a>.</p>
</div>
<div class="section">
<h3><a id="encoding" name="encoding">2.1.2 <tt class="docutils literal"><span class="pre">encoding</span></tt></a></h3>
<p>The encoding of the template file. This is only used for text templates. The
default is to assume “utf-8”.</p>
</div>
<div class="section">
<h3><a id="include-attrs" name="include-attrs">2.1.3 <tt class="docutils literal"><span class="pre">include_attrs</span></tt></a></h3>
<p>Comma-separated list of attribute names that should be considered to have
localizable values. Only used for markup templates.</p>
</div>
<div class="section">
<h3><a id="ignore-tags" name="ignore-tags">2.1.4 <tt class="docutils literal"><span class="pre">ignore_tags</span></tt></a></h3>
<p>Comma-separated list of tag names that should be ignored. Only used for markup
templates.</p>
</div>
<div class="section">
<h3><a id="extract-text" name="extract-text">2.1.5 <tt class="docutils literal"><span class="pre">extract_text</span></tt></a></h3>
<p>Whether text outside explicit <tt class="docutils literal"><span class="pre">gettext</span></tt> function calls should be extracted.
By default, any text nodes not inside ignored tags, and values of attribute in
the <tt class="docutils literal"><span class="pre">include_attrs</span></tt> list are extracted. If this option is disabled, only
strings in <tt class="docutils literal"><span class="pre">gettext</span></tt> function calls are extracted.</p>
<div class="note">
<p class="first admonition-title">Note</p>
<p class="last">If you disable this option, it's not necessary to add the translation
filter as described above. You only need to make sure that the
template has access to the <tt class="docutils literal"><span class="pre">gettext</span></tt> functions it uses.</p>
</div>
</div>
</div>
</div>
<div class="section">
<h1><a id="translation" name="translation">3 Translation</a></h1>
<p>If you have prepared MO files for use with Genshi using the appropriate tools,
you can access the message catalogs with the <a class="reference" href="http://docs.python.org/lib/module-gettext.html">gettext Python module</a>. You'll
probably want to create a <tt class="docutils literal"><span class="pre">gettext.GNUTranslations</span></tt> instance, and make the
translation functions it provides available to your templates by putting them
in the template context.</p>
<p>The <tt class="docutils literal"><span class="pre">Translator</span></tt> filter needs to be added to the filters of the template
(applying it as a stream filter will likely not have the desired effect).
Furthermore it needs to be the first filter in the list, including the internal
filters that Genshi adds itself:</p>
<div class="highlight"><pre><span class="k">from</span> <span class="nn">genshi.filters</span> <span class="k">import</span> <span class="n">Translator</span>
<span class="k">from</span> <span class="nn">genshi.template</span> <span class="k">import</span> <span class="n">MarkupTemplate</span>
<span class="n">template</span> <span class="o">=</span> <span class="n">MarkupTemplate</span><span class="p">(</span><span class="s">"..."</span><span class="p">)</span>
<span class="n">template</span><span class="o">.</span><span class="n">filters</span><span class="o">.</span><span class="n">insert</span><span class="p">(</span><span class="mf">0</span><span class="p">,</span> <span class="n">Translator</span><span class="p">(</span><span class="n">translations</span><span class="o">.</span><span class="n">ugettext</span><span class="p">))</span>
</pre></div>
<p>If you're using <cite>TemplateLoader</cite>, you should specify a callback function in
which you add the filter:</p>
<div class="highlight"><pre><span class="k">from</span> <span class="nn">genshi.filters</span> <span class="k">import</span> <span class="n">Translator</span>
<span class="k">from</span> <span class="nn">genshi.template</span> <span class="k">import</span> <span class="n">TemplateLoader</span>
<span class="k">def</span> <span class="nf">template_loaded</span><span class="p">(</span><span class="n">template</span><span class="p">):</span>
<span class="n">template</span><span class="o">.</span><span class="n">filters</span><span class="o">.</span><span class="n">insert</span><span class="p">(</span><span class="mf">0</span><span class="p">,</span> <span class="n">Translator</span><span class="p">(</span><span class="n">translations</span><span class="o">.</span><span class="n">ugettext</span><span class="p">))</span>
<span class="n">loader</span> <span class="o">=</span> <span class="n">TemplateLoader</span><span class="p">(</span><span class="s">'templates'</span><span class="p">,</span> <span class="n">callback</span><span class="o">=</span><span class="n">template_loaded</span><span class="p">)</span>
<span class="n">template</span> <span class="o">=</span> <span class="n">loader</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="s">"..."</span><span class="p">)</span>
</pre></div>
<p>This approach ensures that the filter is not added everytime the template is
loaded, and thus being applied multiple times.</p>
</div>
<div class="section">
<h1><a id="related-considerations" name="related-considerations">4 Related Considerations</a></h1>
<p>If you intend to produce an application that is fully prepared for an
international audience, there are a couple of other things to keep in mind:</p>
<div class="section">
<h2><a id="unicode" name="unicode">4.1 Unicode</a></h2>
<p>Use <tt class="docutils literal"><span class="pre">unicode</span></tt> internally, not encoded bytestrings. Only encode/decode where
data enters or exits the system. This means that your code works with characters
and not just with bytes, which is an important distinction for example when
calculating the length of a piece of text. When you need to decode/encode, it's
probably a good idea to use UTF-8.</p>
</div>
<div class="section">
<h2><a id="date-and-time" name="date-and-time">4.2 Date and Time</a></h2>
<p>If your application uses datetime information that should be displayed to users
in different timezones, you should try to work with UTC (universal time)
internally. Do the conversion from and to "local time" when the data enters or
exits the system. Make use the Python <a class="reference" href="http://docs.python.org/lib/module-datetime.html">datetime</a> module and the third-party
<a class="reference" href="http://pytz.sourceforge.net/">pytz</a> package.</p>
</div>
<div class="section">
<h2><a id="formatting-and-locale-data" name="formatting-and-locale-data">4.3 Formatting and Locale Data</a></h2>
<p>Make sure you check out the functionality provided by the <a class="reference" href="http://babel.edgewall.org/">Babel</a> project for
things like number and date formatting, locale display strings, etc.</p>
</div>
</div>
<div id="footer">
Visit the Genshi open source project at
<a href="http://genshi.edgewall.org/">http://genshi.edgewall.org/</a>
</div>
</div>
</body>
</html>
distrib
>
Mandriva
>
2010.0
>
i586
>
media
>
contrib-release
>
by-pkgid
>
bd9b8648918182c52f8a2d496fcd571e
>
files
>
182
