<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
<title>GAP (GAPDoc) - Chapter 6: String and Text Utilities</title>
<meta http-equiv="content-type" content="text/html; charset=UTF-8" />
<meta name="generator" content="GAPDoc2HTML" />
<link rel="stylesheet" type="text/css" href="manual.css" />
</head>
<body>
<div class="chlinktop"><span class="chlink1">Goto Chapter: </span><a href="chap0.html">Top</a> <a href="chap1.html">1</a> <a href="chap2.html">2</a> <a href="chap3.html">3</a> <a href="chap4.html">4</a> <a href="chap5.html">5</a> <a href="chap6.html">6</a> <a href="chap7.html">7</a> <a href="chapA.html">A</a> <a href="chapB.html">B</a> <a href="chapC.html">C</a> <a href="chapBib.html">Bib</a> <a href="chapInd.html">Ind</a> </div>
<div class="chlinkprevnexttop"> <a href="chap0.html">Top of Book</a> <a href="chap5.html">Previous Chapter</a> <a href="chap7.html">Next Chapter</a> </div>
<p><a id="X86CEF540862EE042" name="X86CEF540862EE042"></a></p>
<div class="ChapSects"><a href="chap6.html#X86CEF540862EE042">6 <span class="Heading">String and Text Utilities</span></a>
<div class="ContSect"><span class="nocss"> </span><a href="chap6.html#X847DA07C7C46B38A">6.1 <span class="Heading">Text Utilities</span></a>
<span class="ContSS"><br /><span class="nocss"> </span><a href="chap6.html#X786D477C7AB636AA">6.1-1 WHITESPACE</a></span>
<span class="ContSS"><br /><span class="nocss"> </span><a href="chap6.html#X785F61E77899580E">6.1-2 TextAttr</a></span>
<span class="ContSS"><br /><span class="nocss"> </span><a href="chap6.html#X7B8AD7517E5FD0EA">6.1-3 WrapTextAttribute</a></span>
<span class="ContSS"><br /><span class="nocss"> </span><a href="chap6.html#X812058CE7C8E9022">6.1-4 FormatParagraph</a></span>
<span class="ContSS"><br /><span class="nocss"> </span><a href="chap6.html#X82A9121678923445">6.1-5 SubstitutionSublist</a></span>
<span class="ContSS"><br /><span class="nocss"> </span><a href="chap6.html#X83DE31017B557136">6.1-6 StripBeginEnd</a></span>
<span class="ContSS"><br /><span class="nocss"> </span><a href="chap6.html#X7A5978CF84C3C2D3">6.1-7 StripEscapeSequences</a></span>
<span class="ContSS"><br /><span class="nocss"> </span><a href="chap6.html#X7D71CB837EE969D4">6.1-8 RepeatedString</a></span>
<span class="ContSS"><br /><span class="nocss"> </span><a href="chap6.html#X7CEEA5B57D7BB38F">6.1-9 NumberDigits</a></span>
<span class="ContSS"><br /><span class="nocss"> </span><a href="chap6.html#X7AF694D9839BF65C">6.1-10 PositionMatchingDelimiter</a></span>
<span class="ContSS"><br /><span class="nocss"> </span><a href="chap6.html#X832556617F10AAA8">6.1-11 WordsString</a></span>
<span class="ContSS"><br /><span class="nocss"> </span><a href="chap6.html#X83F2821783DA9826">6.1-12 Base64String</a></span>
</div>
<div class="ContSect"><span class="nocss"> </span><a href="chap6.html#X8489C67D80399814">6.2 <span class="Heading">Unicode Strings</span></a>
<span class="ContSS"><br /><span class="nocss"> </span><a href="chap6.html#X8475671278948DDD">6.2-1 <span class="Heading">Unicode Strings and Characters</span></a>
</span>
<span class="ContSS"><br /><span class="nocss"> </span><a href="chap6.html#X818A31567EB30A39">6.2-2 Encode</a></span>
<span class="ContSS"><br /><span class="nocss"> </span><a href="chap6.html#X801237207E06A876">6.2-3 <span class="Heading">Lengths of UTF-8 strings</span></a>
</span>
</div>
<div class="ContSect"><span class="nocss"> </span><a href="chap6.html#X860C83047DC4F1BC">6.3 <span class="Heading">Print Utilities</span></a>
<span class="ContSS"><br /><span class="nocss"> </span><a href="chap6.html#X8603B90C7C3F0AB1">6.3-1 PrintTo1</a></span>
<span class="ContSS"><br /><span class="nocss"> </span><a href="chap6.html#X829B720C86E57E8B">6.3-2 StringPrint</a></span>
<span class="ContSS"><br /><span class="nocss"> </span><a href="chap6.html#X812A8326844BC910">6.3-3 PrintFormattedString</a></span>
<span class="ContSS"><br /><span class="nocss"> </span><a href="chap6.html#X7BB6731F7E3AAA98">6.3-4 Page</a></span>
<span class="ContSS"><br /><span class="nocss"> </span><a href="chap6.html#X7E14D32181FBC3C3">6.3-5 StringFile</a></span>
</div>
</div>
<h3>6 <span class="Heading">String and Text Utilities</span></h3>
<p><a id="X847DA07C7C46B38A" name="X847DA07C7C46B38A"></a></p>
<h4>6.1 <span class="Heading">Text Utilities</span></h4>
<p>This section describes some utility functions for handling texts within <strong class="pkg">GAP</strong>. They are used by the functions in the <strong class="pkg">GAPDoc</strong> package but may be useful for other purposes as well. We start with some variables containing useful strings and go on with functions for parsing and reformatting text.</p>
<p><a id="X786D477C7AB636AA" name="X786D477C7AB636AA"></a></p>
<h5>6.1-1 WHITESPACE</h5>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> WHITESPACE</code></td><td class="tdright">( global variable )</td></tr></table></div>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> CAPITALLETTERS</code></td><td class="tdright">( global variable )</td></tr></table></div>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> SMALLLETTERS</code></td><td class="tdright">( global variable )</td></tr></table></div>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> LETTERS</code></td><td class="tdright">( global variable )</td></tr></table></div>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> DIGITS</code></td><td class="tdright">( global variable )</td></tr></table></div>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> HEXDIGITS</code></td><td class="tdright">( global variable )</td></tr></table></div>
<p>These variables contain sets of characters which are useful for text processing. They are defined as follows.</p>
<dl>
<dt><strong class="Mark"><code class="code">WHITESPACE</code></strong></dt>
<dd><p><code class="code">" \n\t\r"</code></p>
</dd>
<dt><strong class="Mark"><code class="code">CAPITALLETTERS</code></strong></dt>
<dd><p><code class="code">"ABCDEFGHIJKLMNOPQRSTUVWXYZ"</code></p>
</dd>
<dt><strong class="Mark"><code class="code">SMALLLETTERS</code></strong></dt>
<dd><p><code class="code">"abcdefghijklmnopqrstuvwxyz"</code></p>
</dd>
<dt><strong class="Mark"><code class="code">LETTERS</code></strong></dt>
<dd><p>concatenation of <code class="code">CAPITALLETTERS</code> and <code class="code">SMALLLETTERS</code></p>
</dd>
<dt><strong class="Mark"><code class="code">DIGITS</code></strong></dt>
<dd><p><code class="code">"0123456789"</code></p>
</dd>
<dt><strong class="Mark"><code class="code">HEXDIGITS</code></strong></dt>
<dd><p><code class="code">"0123456789ABCDEFabcdef"</code></p>
</dd>
</dl>
<p><a id="X785F61E77899580E" name="X785F61E77899580E"></a></p>
<h5>6.1-2 TextAttr</h5>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> TextAttr</code></td><td class="tdright">( global variable )</td></tr></table></div>
<p>The record <code class="func">TextAttr</code> contains strings which can be printed to change the terminal attribute for the following characters. This only works with terminals which understand basic ANSI escape sequences. Try the following example to see if this is the case for the terminal you are using. It shows the effect of the foreground and background color attributes and of the <code class="code">.bold</code>, <code class="code">.blink</code>, <code class="code">.normal</code>, <code class="code">.reverse</code> and<code class="code">.underscore</code> which can partly be mixed.</p>
<table class="example">
<tr><td><pre>
extra := ["CSI", "reset", "delline", "home"];;
for t in Difference(RecNames(TextAttr), extra) do
Print(TextAttr.(t), "TextAttr.", t, TextAttr.reset,"\n");
od;
</pre></td></tr></table>
<p>The suggested defaults for colors <code class="code">0..7</code> are black, red, green, brown, blue, magenta, cyan, white. But this may be different for your terminal configuration.</p>
<p>The escape sequence <code class="code">.delline</code> deletes the content of the current line and <code class="code">.home</code> moves the cursor to the beginning of the current line.</p>
<table class="example">
<tr><td><pre>
for i in [1..5] do
Print(TextAttr.home, TextAttr.delline, String(i,-6), "\c");
Sleep(1);
od;
</pre></td></tr></table>
<p>Whenever you use this in some printing routines you should make it optional. Use these attributes only, when the variable <code class="code">ANSI_COLORS</code> has the value <code class="keyw">true</code>.</p>
<p><a id="X7B8AD7517E5FD0EA" name="X7B8AD7517E5FD0EA"></a></p>
<h5>6.1-3 WrapTextAttribute</h5>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> WrapTextAttribute</code>( <var class="Arg">str, attr</var> )</td><td class="tdright">( function )</td></tr></table></div>
<p><b>Returns: </b>a string with markup</p>
<p>The argument <var class="Arg">str</var> must be a text as <strong class="pkg">GAP</strong> string, possibly with markup by escape sequences as in <code class="func">TextAttr</code> (<a href="chap6.html#X785F61E77899580E"><b>6.1-2</b></a>). This function returns a string which is wrapped by the escape sequences <var class="Arg">attr</var> and <code class="code">TextAttr.reset</code>. It takes care of markup in the given string by appending <var class="Arg">attr</var> also after each given <code class="code">TextAttr.reset</code> in <var class="Arg">str</var>.</p>
<table class="example">
<tr><td><pre>
gap> str := Concatenation("XXX",TextAttr.2, "BLUB", TextAttr.reset,"YYY");
"XXX\033[32mBLUB\033[0mYYY"
gap> str2 := WrapTextAttribute(str, TextAttr.1);
"\033[31mXXX\033[32mBLUB\033[0m\033[31mYYY\033[0m"
gap> str3 := WrapTextAttribute(str, TextAttr.underscore);
"\033[4mXXX\033[32mBLUB\033[0m\033[4mYYY\033[0m"
gap> # use Print(str); and so on to see how it looks like.
</pre></td></tr></table>
<p><a id="X812058CE7C8E9022" name="X812058CE7C8E9022"></a></p>
<h5>6.1-4 FormatParagraph</h5>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> FormatParagraph</code>( <var class="Arg">str[, len][, flush][, attr][, widthfun], ]]</var> )</td><td class="tdright">( function )</td></tr></table></div>
<p><b>Returns: </b>the formatted paragraph as string</p>
<p>This function formats a text given in the string <var class="Arg">str</var> as a paragraph. The optional arguments have the following meaning:</p>
<dl>
<dt><strong class="Mark"><var class="Arg">len</var></strong></dt>
<dd><p>the length of the lines of the resulting text (default is <code class="code">78</code>)</p>
</dd>
<dt><strong class="Mark"><var class="Arg">flush</var></strong></dt>
<dd><p>can be <code class="code">"left"</code>, <code class="code">"right"</code>, <code class="code">"center"</code> or <code class="code">"both"</code>, telling that lines should be flushed left, flushed right, centered or left-right justified, respectively (default is <code class="code">"both"</code>)</p>
</dd>
<dt><strong class="Mark"><var class="Arg">attr</var></strong></dt>
<dd><p>is a list of two strings; the first is prepended and the second appended to each line of the result (can for example be used for indenting, <code class="code">[" ", ""]</code>, or some markup, <code class="code">[TextAttr.bold, TextAttr.reset]</code>, default is <code class="code">["", ""]</code>)</p>
</dd>
<dt><strong class="Mark"><var class="Arg">widthfun</var></strong></dt>
<dd><p>must be a function which returns the display width of text in <var class="Arg">str</var>. The default is <code class="code">Length</code> assuming that each byte corresponds to a character of width one. If <var class="Arg">str</var> is given in <code class="code">UTF-8</code> encoding one can use <code class="func">WidthUTF8String</code> (<a href="chap6.html#X801237207E06A876"><b>6.2-3</b></a>) here.</p>
</dd>
</dl>
<p>This function tries to handle markup with the escape sequences explained in <code class="func">TextAttr</code> (<a href="chap6.html#X785F61E77899580E"><b>6.1-2</b></a>) correctly.</p>
<table class="example">
<tr><td><pre>
gap> str := "One two three four five six seven eight nine ten eleven.";;
gap> Print(FormatParagraph(str, 25, "left", ["/* ", " */"]));
/* One two three four five */
/* six seven eight nine ten */
/* eleven. */
</pre></td></tr></table>
<p><a id="X82A9121678923445" name="X82A9121678923445"></a></p>
<h5>6.1-5 SubstitutionSublist</h5>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> SubstitutionSublist</code>( <var class="Arg">list, sublist, new[, flag]</var> )</td><td class="tdright">( function )</td></tr></table></div>
<p><b>Returns: </b>the changed list</p>
<p>This function looks for (non-overlapping) occurrences of a sublist <var class="Arg">sublist</var> in a list <var class="Arg">list</var> (compare <code class="func">PositionSublist</code> (<a href="../../../doc/htm/ref/CHAP021.htm#SECT016"><b>Reference: PositionSublist</b></a>)) and returns a list where these are substituted with the list <var class="Arg">new</var>.</p>
<p>The optional argument <var class="Arg">flag</var> can either be <code class="code">"all"</code> (this is the default if not given) or <code class="code">"one"</code>. In the second case only the first occurrence of <var class="Arg">sublist</var> is substituted.</p>
<p>If <var class="Arg">sublist</var> does not occur in <var class="Arg">list</var> then <var class="Arg">list</var> itself is returned (and not a <code class="code">ShallowCopy(list)</code>).</p>
<table class="example">
<tr><td><pre>
gap> SubstitutionSublist("xababx", "ab", "a");
"xaax"
</pre></td></tr></table>
<p><a id="X83DE31017B557136" name="X83DE31017B557136"></a></p>
<h5>6.1-6 StripBeginEnd</h5>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> StripBeginEnd</code>( <var class="Arg">list, strip</var> )</td><td class="tdright">( function )</td></tr></table></div>
<p><b>Returns: </b>changed string</p>
<p>Here <var class="Arg">list</var> and <var class="Arg">strip</var> must be lists. This function returns the sublist of list which does not contain the leading and trailing entries which are entries of <var class="Arg">strip</var>. If the result is equal to <var class="Arg">list</var> then <var class="Arg">list</var> itself is returned.</p>
<table class="example">
<tr><td><pre>
gap> StripBeginEnd(" ,a, b,c, ", ", ");
"a, b,c"
</pre></td></tr></table>
<p><a id="X7A5978CF84C3C2D3" name="X7A5978CF84C3C2D3"></a></p>
<h5>6.1-7 StripEscapeSequences</h5>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> StripEscapeSequences</code>( <var class="Arg">str</var> )</td><td class="tdright">( function )</td></tr></table></div>
<p><b>Returns: </b>string without escape sequences</p>
<p>This function returns the string one gets from the string <var class="Arg">str</var> by removing all escape sequences which are explained in <code class="func">TextAttr</code> (<a href="chap6.html#X785F61E77899580E"><b>6.1-2</b></a>). If <var class="Arg">str</var> does not contain such a sequence then <var class="Arg">str</var> itself is returned.</p>
<p><a id="X7D71CB837EE969D4" name="X7D71CB837EE969D4"></a></p>
<h5>6.1-8 RepeatedString</h5>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> RepeatedString</code>( <var class="Arg">c, len</var> )</td><td class="tdright">( function )</td></tr></table></div>
<p>Here <var class="Arg">c</var> must be either a character or a string and <var class="Arg">len</var> is a non-negative number. Then <code class="func">RepeatedString</code> returns a string of length <var class="Arg">len</var> consisting of copies of <var class="Arg">c</var>.</p>
<table class="example">
<tr><td><pre>
gap> RepeatedString('=',51);
"==================================================="
gap> RepeatedString("*=",51);
"*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*"
</pre></td></tr></table>
<p><a id="X7CEEA5B57D7BB38F" name="X7CEEA5B57D7BB38F"></a></p>
<h5>6.1-9 NumberDigits</h5>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> NumberDigits</code>( <var class="Arg">str, base</var> )</td><td class="tdright">( function )</td></tr></table></div>
<p><b>Returns: </b>integer</p>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> DigitsNumber</code>( <var class="Arg">n, base</var> )</td><td class="tdright">( function )</td></tr></table></div>
<p><b>Returns: </b>string</p>
<p>The argument <var class="Arg">str</var> of <code class="func">NumberDigits</code> must be a string consisting only of an optional leading <code class="code">'-'</code> and characters in <code class="code">0123456789abcdefABCDEF</code>, describing an integer in base <var class="Arg">base</var> with 2 <= <var class="Arg">base</var> <= 16. This function returns the corresponding integer.</p>
<p>The function <code class="func">DigitsNumber</code> does the reverse.</p>
<table class="example">
<tr><td><pre>
gap> NumberDigits("1A3F",16);
6719
gap> DigitsNumber(6719, 16);
"1A3F"
</pre></td></tr></table>
<p><a id="X7AF694D9839BF65C" name="X7AF694D9839BF65C"></a></p>
<h5>6.1-10 PositionMatchingDelimiter</h5>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> PositionMatchingDelimiter</code>( <var class="Arg">str, delim, pos</var> )</td><td class="tdright">( function )</td></tr></table></div>
<p><b>Returns: </b>position as integer or <code class="keyw">fail</code></p>
<p>Here <var class="Arg">str</var> must be a string and <var class="Arg">delim</var> a string with two different characters. This function searches the smallest position <code class="code">r</code> of the character <code class="code"><var class="Arg">delim</var>[2]</code> in <var class="Arg">str</var> such that the number of occurrences of <code class="code"><var class="Arg">delim</var>[2]</code> in <var class="Arg">str</var> between positions <code class="code"><var class="Arg">pos</var>+1</code> and <code class="code">r</code> is by one greater than the corresponding number of occurrences of <code class="code"><var class="Arg">delim</var>[1]</code>.</p>
<p>If such an <code class="code">r</code> exists, it is returned. Otherwise <code class="keyw">fail</code> is returned.</p>
<table class="example">
<tr><td><pre>
gap> PositionMatchingDelimiter("{}x{ab{c}d}", "{}", 0);
fail
gap> PositionMatchingDelimiter("{}x{ab{c}d}", "{}", 1);
2
gap> PositionMatchingDelimiter("{}x{ab{c}d}", "{}", 6);
11
</pre></td></tr></table>
<p><a id="X832556617F10AAA8" name="X832556617F10AAA8"></a></p>
<h5>6.1-11 WordsString</h5>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> WordsString</code>( <var class="Arg">str</var> )</td><td class="tdright">( function )</td></tr></table></div>
<p><b>Returns: </b>list of strings containing the words</p>
<p>This returns the list of words of a text stored in the string <var class="Arg">str</var>. All non-letters are considered as word boundaries and are removed.</p>
<table class="example">
<tr><td><pre>
gap> WordsString("one_two \n three!?");
[ "one", "two", "three" ]
</pre></td></tr></table>
<p><a id="X83F2821783DA9826" name="X83F2821783DA9826"></a></p>
<h5>6.1-12 Base64String</h5>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> Base64String</code>( <var class="Arg">str</var> )</td><td class="tdright">( function )</td></tr></table></div>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> StringBase64</code>( <var class="Arg">bstr</var> )</td><td class="tdright">( function )</td></tr></table></div>
<p><b>Returns: </b>a string</p>
<p>The first function translates arbitrary binary data given as a GAP string into a <em>base 64</em> encoded string. This encoded string contains only printable ASCII characters and is used in various data transfer protocols (<code class="code">MIME</code> encoded emails, weak password encryption, ...). We use the specification in <span class="URL"><a href="http://tools.ietf.org/html/rfc2045">RFC 2045</a></span>.</p>
<p>The second function has the reverse functionality. Here we also accept the characters <code class="code">-_</code> instead of <code class="code">+/</code> as last two characters. Whitespace is ignored.</p>
<table class="example">
<tr><td><pre>
gap> b := Base64String("This is a secret!");
"VGhpcyBpcyBhIHNlY3JldCEA="
gap> StringBase64(b);
"This is a secret!"
</pre></td></tr></table>
<p><a id="X8489C67D80399814" name="X8489C67D80399814"></a></p>
<h4>6.2 <span class="Heading">Unicode Strings</span></h4>
<p>The <strong class="pkg">GAPDoc</strong> package provides some tools to deal with unicode characters and strings. These can be used for recoding text strings between various encodings.</p>
<p><a id="X8475671278948DDD" name="X8475671278948DDD"></a></p>
<h5>6.2-1 <span class="Heading">Unicode Strings and Characters</span></h5>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> Unicode</code>( <var class="Arg">list[, encoding]</var> )</td><td class="tdright">( operation )</td></tr></table></div>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> UChar</code>( <var class="Arg">num</var> )</td><td class="tdright">( operation )</td></tr></table></div>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> IsUnicodeString</code></td><td class="tdright">( filter )</td></tr></table></div>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> IsUnicodeCharacter</code></td><td class="tdright">( filter )</td></tr></table></div>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> IntListUnicodeString</code>( <var class="Arg">ustr</var> )</td><td class="tdright">( function )</td></tr></table></div>
<p>Unicode characters are described by their <em>codepoint</em>, an integer in the range from 0 to 2^21-1. For details about unicode, see <span class="URL"><a href="http://www.unicode.org">http://www.unicode.org</a></span>.</p>
<p>The function <code class="func">UChar</code> wraps an integer <var class="Arg">num</var> into a <strong class="pkg">GAP</strong> object lying in the filter <code class="func">IsUnicodeCharacter</code>. Use <code class="code">Int</code> to get the codepoint back. The argument <var class="Arg">num</var> can also be a <strong class="pkg">GAP</strong> character which is then translated to an integer via <code class="func">INT_CHAR</code> (<a href="../../../doc/htm/ref/CHAP026.htm#SECT006"><b>Reference: INT_CHAR</b></a>).</p>
<p><code class="func">Unicode</code> produces a <strong class="pkg">GAP</strong> object in the filter <code class="func">IsUnicodeString</code>. This is a wrapped list of integers for the unicode characters in the string. The function <code class="func">IntListUnicodeString</code> gives access to this list of integers. Basic list functionality is available for <code class="func">IsUnicodeString</code> elements. The entries are in <code class="func">IsUnicodeCharacter</code>. The argument <var class="Arg">list</var> for <code class="func">Unicode</code> is either a list of integers or a <strong class="pkg">GAP</strong> string. In the latter case an <var class="Arg">encoding</var> can be specified as string, its default is <code class="code">"UTF-8"</code>.</p>
<p>Currently supported encodings can be found in <code class="code">UNICODE_RECODE.NormalizedEncodings</code> (ASCII, ISO-8859-X, UTF-8 and aliases). The encoding <code class="code">"XML"</code> means an ASCII encoding in which non-ASCII characters are specified by XML character entities. The encoding <code class="code">"URL"</code> is for URL-encoded (also called percent-encoded strings, as specified in RFC 3986 (<span class="URL"><a href="http://www.ietf.org/rfc/rfc3986.txt">see here</a></span>). The listed encodings <code class="code">"LaTeX"</code> and aliases cannot be used with <code class="func">Unicode</code>. See the operation <code class="func">Encode</code> (<a href="chap6.html#X818A31567EB30A39"><b>6.2-2</b></a>) for mapping a unicode string to a <strong class="pkg">GAP</strong> string.</p>
<table class="example">
<tr><td><pre>
gap> ustr := Unicode("a and \366", "latin1");
Unicode("a and ö")
gap> ustr = Unicode("a and &#246;", "XML");
true
gap> IntListUnicodeString(ustr);
[ 97, 32, 97, 110, 100, 32, 246 ]
gap> ustr[7];
'ö'
</pre></td></tr></table>
<p><a id="X818A31567EB30A39" name="X818A31567EB30A39"></a></p>
<h5>6.2-2 Encode</h5>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> Encode</code>( <var class="Arg">ustr[, encoding]</var> )</td><td class="tdright">( operation )</td></tr></table></div>
<p><b>Returns: </b>a <strong class="pkg">GAP</strong> string</p>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> SimplifiedUnicodeString</code>( <var class="Arg">ustr[, encoding][, "single"]</var> )</td><td class="tdright">( function )</td></tr></table></div>
<p><b>Returns: </b>a unicode string</p>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> LowercaseUnicodeString</code>( <var class="Arg">ustr</var> )</td><td class="tdright">( function )</td></tr></table></div>
<p><b>Returns: </b>a unicode string</p>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> UppercaseUnicodeString</code>( <var class="Arg">ustr</var> )</td><td class="tdright">( function )</td></tr></table></div>
<p><b>Returns: </b>a unicode string</p>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> LaTeXUnicodeTable</code></td><td class="tdright">( global variable )</td></tr></table></div>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> SimplifiedUnicodeTable</code></td><td class="tdright">( global variable )</td></tr></table></div>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> LowercaseUnicodeTable</code></td><td class="tdright">( global variable )</td></tr></table></div>
<p>The operation <code class="func">Encode</code> translates a unicode string <var class="Arg">ustr</var> into a <strong class="pkg">GAP</strong> string in some specified <var class="Arg">encoding</var>. The default encoding is <code class="code">"UTF-8"</code>.</p>
<p>Supported encodings can be found in <code class="code">UNICODE_RECODE.NormalizedEncodings</code>. Except for some cases mentioned below characters which are not available in the target encoding are substituted by '?' characters.</p>
<p>If the <var class="Arg">encoding</var> is <code class="code">"URL"</code> (see <code class="func">Unicode</code> (<a href="chap6.html#X8475671278948DDD"><b>6.2-1</b></a>)) then an optional argument <var class="Arg">encreserved</var> can be given, it must be a list of reserved characters which should be percent encoded; the default is to encode only the <code class="code">%</code> character.</p>
<p>The encoding <code class="code">"LaTeX"</code> substitutes non-ASCII characters and LaTeX special characters by LaTeX code as given in an ordered list <code class="code">LaTeXUnicodeTable</code> of pairs [codepoint, string]. If you have a unicode character for which no substitution is contained in that list, you will get a warning and the translation is <code class="code">Unicode(nr)</code>. In this case find a substitution and add a corresponding [codepoint, string] pair to <code class="code">LaTeXUnicodeTable</code> using <code class="func">AddSet</code> (<a href="../../../doc/htm/ref/CHAP021.htm#SECT019"><b>Reference: AddSet</b></a>). Also, please, tell the <strong class="pkg">GAPDoc</strong> authors about your addition, such that we can extend the list <code class="code">LaTeXUnicodeTable</code>. (Most of the initial entries were generated from lists in the TeX projects encTeX and <code class="code">ucs</code>.) There are some variants of this encoding:</p>
<p><code class="code">"LaTeXleavemarkup"</code> does the same translations for non-ASCII characters but leaves the LaTeX special characters (e.g., any LaTeX commands) as they are.</p>
<p><code class="code">"LaTeXUTF8"</code> does not give a warning about unicode characters without explicit translation, instead it translates the character to its <code class="code">UTF-8</code> encoding. Make sure to setup your LaTeX document such that all these characters are understood.</p>
<p><code class="code">"LaTeXUTF8leavemarkup"</code> is a combination of the last two variants.</p>
<p>Note that the <code class="code">"LaTeX"</code> encoding can only be used with <code class="func">Encode</code> but not for the opposite translation with <code class="func">Unicode</code> (<a href="chap6.html#X8475671278948DDD"><b>6.2-1</b></a>) (which would need far too complicated heuristics).</p>
<p>The function <code class="func">SimplifiedUnicodeString</code> can be used to substitute many non-ASCII characters by related ASCII characters or strings (e.g., by a corresponding character without accents). The argument <var class="Arg">ustr</var> and the result are unicode strings, if <var class="Arg">encoding</var> is <code class="code">"ASCII"</code> then all non-ASCII characters are translated, otherwise only the non-latin1 characters. If the string <code class="code">"single"</code> in an argument then only substitutions are considered which don't make the result string longer. The translations are stored in a sorted list <code class="code">SimplifiedUnicodeTable</code>. Its entries are of the form <code class="code">[codepoint, trans1, trans2, ...]</code>. Here <code class="code">trans1</code> and so on is either an integer for the codepoint of a substitution character or it is a list of codepoint integers. If you are missing characters in this list and know a sensible ASCII approximation, then add an entry (with <code class="func">AddSet</code> (<a href="../../../doc/htm/ref/CHAP021.htm#SECT019"><b>Reference: AddSet</b></a>)) and tell the <strong class="pkg">GAPDoc</strong> authors about it. (The initial content of <code class="code">SimplifiedUnicodeTable</code> was mainly generated from the "<code class="code">transtab</code>" tables by Markus Kuhn.)</p>
<p>The function <code class="func">LowercaseUnicodeString</code> gets and returns a unicode string and translates each uppercase character to its corresponding lowercase version. This function uses a list <code class="code">LowercaseUnicodeTable</code> of pairs of codepoint integers. This list was generated using the file <code class="file">UnicodeData.txt</code> from the unicode definition (field 14 in each row).</p>
<p>The function <code class="func">UppercaseUnicodeString</code> does the similar translation to uppercase characters.</p>
<table class="example">
<tr><td><pre>
gap> ustr := Unicode("a and &#246;", "XML");
Unicode("a and ö")
gap> SimplifiedUnicodeString(ustr, "ASCII");
Unicode("a and oe")
gap> SimplifiedUnicodeString(ustr, "ASCII", "single");
Unicode("a and o")
gap> ustr2 := UppercaseUnicodeString(ustr);;
gap> Print(Encode(ustr2, GAPInfo.TermEncoding), "\n");
A AND Ö
</pre></td></tr></table>
<p><a id="X801237207E06A876" name="X801237207E06A876"></a></p>
<h5>6.2-3 <span class="Heading">Lengths of UTF-8 strings</span></h5>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> WidthUTF8String</code>( <var class="Arg">str</var> )</td><td class="tdright">( function )</td></tr></table></div>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> NrCharsUTF8String</code>( <var class="Arg">str</var> )</td><td class="tdright">( function )</td></tr></table></div>
<p><b>Returns: </b>an integer</p>
<p>Let <var class="Arg">str</var> be a <strong class="pkg">GAP</strong> string with text in UTF-8 encoding. There are three "lengths" of such a string which must be distinguished. The operation <code class="func">Length</code> (<a href="../../../doc/htm/ref/CHAP021.htm#SECT017"><b>Reference: Length</b></a>) returns the number of bytes and so the memory occupied by <var class="Arg">str</var>. The function <code class="func">NrCharsUTF8String</code> returns the number of unicode characters in <var class="Arg">str</var>, that is the length of <code class="code">Unicode(<var class="Arg">str</var>)</code>.</p>
<p>In many applications the function <code class="func">WidthUTF8String</code> is more interesting, it returns the number of columns needed by the string if printed to a terminal. This takes into account that some unicode characters are combining characters and that there are wide characters which need two columns (e.g., for Chinese or Japanese). (To be precise: This implementation assumes that there are no control characters in <var class="Arg">str</var> and uses the character width returned by the <code class="code">wcwidth</code> function in the GNU C-library called with UTF-8 locale.)</p>
<table class="example">
<tr><td><pre>
gap> # A, German umlaut u, B, zero width space, C, newline
gap> str := Encode( Unicode( "A&#xFC;B&#x200B;C\n", "XML" ) );;
gap> Print(str);
AüBC
gap> # umlaut u needs two bytes and the zero width space three
gap> Length(str);
9
gap> NrCharsUTF8String(str);
6
gap> # zero width space and newline don't contribute to width
gap> WidthUTF8String(str);
4
</pre></td></tr></table>
<p><a id="X860C83047DC4F1BC" name="X860C83047DC4F1BC"></a></p>
<h4>6.3 <span class="Heading">Print Utilities</span></h4>
<p>The following printing utilities turned out to be useful for interactive work with texts in <strong class="pkg">GAP</strong>. But they are more general and so we document them here.</p>
<p><a id="X8603B90C7C3F0AB1" name="X8603B90C7C3F0AB1"></a></p>
<h5>6.3-1 PrintTo1</h5>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> PrintTo1</code>( <var class="Arg">filename, fun</var> )</td><td class="tdright">( function )</td></tr></table></div>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> AppendTo1</code>( <var class="Arg">filename, fun</var> )</td><td class="tdright">( function )</td></tr></table></div>
<p>The argument <var class="Arg">fun</var> must be a function without arguments. Everything which is printed by a call <var class="Arg">fun()</var> is printed into the file <var class="Arg">filename</var>. As with <code class="func">PrintTo</code> (<a href="../../../doc/htm/ref/CHAP009.htm#SECT007"><b>Reference: PrintTo</b></a>) and <code class="func">AppendTo</code> (<a href="../../../doc/htm/ref/CHAP009.htm#SECT007"><b>Reference: AppendTo</b></a>) this overwrites or appends to, respectively, a previous content of <var class="Arg">filename</var>.</p>
<p>These functions can be particularly efficient when many small pieces of text shall be written to a file, because no multiple reopening of the file is necessary.</p>
<table class="example">
<tr><td><pre>
gap> f := function() local i;
> for i in [1..100000] do Print(i, "\n"); od; end;;
gap> PrintTo1("nonsense", f); # now check the local file `nonsense'
</pre></td></tr></table>
<p><a id="X829B720C86E57E8B" name="X829B720C86E57E8B"></a></p>
<h5>6.3-2 StringPrint</h5>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> StringPrint</code>( <var class="Arg">obj1[, obj2[, ...]]</var> )</td><td class="tdright">( function )</td></tr></table></div>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> StringView</code>( <var class="Arg">obj</var> )</td><td class="tdright">( function )</td></tr></table></div>
<p>These functions return a string containing the output of a <code class="code">Print</code> or <code class="code">ViewObj</code> call with the same arguments.</p>
<p>This should be considered as a (temporary?) hack. It would be better to have <code class="func">String</code> (<a href="../../../doc/htm/ref/CHAP026.htm#SECT005"><b>Reference: String</b></a>) methods for all <strong class="pkg">GAP</strong> objects and to have a generic <code class="func">Print</code> (<a href="../../../doc/htm/ref/CHAP006.htm#SECT003"><b>Reference: Print</b></a>)-function which just interprets these strings.</p>
<p><a id="X812A8326844BC910" name="X812A8326844BC910"></a></p>
<h5>6.3-3 PrintFormattedString</h5>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> PrintFormattedString</code>( <var class="Arg">str</var> )</td><td class="tdright">( function )</td></tr></table></div>
<p>This function prints a string <var class="Arg">str</var>. The difference to <code class="code">Print(str);</code> is that no additional line breaks are introduced by <strong class="pkg">GAP</strong>'s standard printing mechanism. This can be used to print lines which are longer than the current screen width. In particular one can print text which contains escape sequences like those explained in <code class="func">TextAttr</code> (<a href="chap6.html#X785F61E77899580E"><b>6.1-2</b></a>), where lines may have more characters than <em>visible characters</em>.</p>
<p><a id="X7BB6731F7E3AAA98" name="X7BB6731F7E3AAA98"></a></p>
<h5>6.3-4 Page</h5>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> Page</code>( <var class="Arg">...</var> )</td><td class="tdright">( function )</td></tr></table></div>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> PageDisplay</code>( <var class="Arg">obj</var> )</td><td class="tdright">( function )</td></tr></table></div>
<p>These functions are similar to <code class="func">Print</code> (<a href="../../../doc/htm/ref/CHAP006.htm#SECT003"><b>Reference: Print</b></a>) and <code class="func">Display</code> (<a href="../../../doc/htm/ref/CHAP006.htm#SECT003"><b>Reference: Display</b></a>), respectively. The difference is that the output is not sent directly to the screen, but is piped into the current pager; see <code class="func">PAGER</code> (<a href="../../../doc/htm/ref/CHAP002.htm#SECT004"><b>Reference: Pager</b></a>).</p>
<table class="example">
<tr><td><pre>
gap> Page([1..1421]+0);
gap> PageDisplay(CharacterTable("Symmetric", 14));
</pre></td></tr></table>
<p><a id="X7E14D32181FBC3C3" name="X7E14D32181FBC3C3"></a></p>
<h5>6.3-5 StringFile</h5>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> StringFile</code>( <var class="Arg">filename</var> )</td><td class="tdright">( function )</td></tr></table></div>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">> FileString</code>( <var class="Arg">filename, str[, append]</var> )</td><td class="tdright">( function )</td></tr></table></div>
<p>The function <code class="func">StringFile</code> returns the content of file <var class="Arg">filename</var> as a string. This works efficiently with arbitrary (binary or text) files. If something went wrong, this function returns <code class="keyw">fail</code>.</p>
<p>Conversely the function <code class="func">FileString</code> writes the content of a string <var class="Arg">str</var> into the file <var class="Arg">filename</var>. If the optional third argument <var class="Arg">append</var> is given and equals <code class="keyw">true</code> then the content of <var class="Arg">str</var> is appended to the file. Otherwise previous content of the file is deleted. This function returns the number of bytes written or <code class="keyw">fail</code> if something went wrong.</p>
<p>Both functions are quite efficient, even with large files.</p>
<div class="chlinkprevnextbot"> <a href="chap0.html">Top of Book</a> <a href="chap5.html">Previous Chapter</a> <a href="chap7.html">Next Chapter</a> </div>
<div class="chlinkbot"><span class="chlink1">Goto Chapter: </span><a href="chap0.html">Top</a> <a href="chap1.html">1</a> <a href="chap2.html">2</a> <a href="chap3.html">3</a> <a href="chap4.html">4</a> <a href="chap5.html">5</a> <a href="chap6.html">6</a> <a href="chap7.html">7</a> <a href="chapA.html">A</a> <a href="chapB.html">B</a> <a href="chapC.html">C</a> <a href="chapBib.html">Bib</a> <a href="chapInd.html">Ind</a> </div>
<hr />
<p class="foot">generated by <a href="http://www.math.rwth-aachen.de/~Frank.Luebeck/GAPDoc">GAPDoc2HTML</a></p>
</body>
</html>
distrib
>
Mandriva
>
2010.0
>
i586
>
media
>
contrib-release
>
by-pkgid
>
91213ddcfbe7f54821d42c2d9e091326
>
files
>
57
