<?xml version="1.0" encoding="utf-8" ?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta name="generator" content="Docutils 0.3.10: http://docutils.sourceforge.net/" /> <title>Using the pyparsing module</title> <meta name="author" content="Paul McGuire" /> <meta name="date" content="April, 2006" /> <meta name="copyright" content="Copyright © 2003-2006 Paul McGuire." /> <style type="text/css"> /* :Author: David Goodger :Contact: goodger@users.sourceforge.net :Date: $Date: 2005-12-18 01:56:14 +0100 (Sun, 18 Dec 2005) $ :Revision: $Revision: 4224 $ :Copyright: This stylesheet has been placed in the public domain. Default cascading style sheet for the HTML output of Docutils. See http://docutils.sf.net/docs/howto/html-stylesheets.html for how to customize this style sheet. */ /* used to remove borders from tables and images */ .borderless, table.borderless td, table.borderless th { border: 0 } table.borderless td, table.borderless th { /* Override padding for "table.docutils td" with "! important". The right padding separates the table cells. */ padding: 0 0.5em 0 0 ! important } .first { /* Override more specific margin styles with "! important". */ margin-top: 0 ! important } .last, .with-subtitle { margin-bottom: 0 ! important } .hidden { display: none } a.toc-backref { text-decoration: none ; color: black } blockquote.epigraph { margin: 2em 5em ; } dl.docutils dd { margin-bottom: 0.5em } /* Uncomment (and remove this text!) to get bold-faced definition list terms dl.docutils dt { font-weight: bold } */ div.abstract { margin: 2em 5em } div.abstract p.topic-title { font-weight: bold ; text-align: center } div.admonition, div.attention, div.caution, div.danger, div.error, div.hint, div.important, div.note, div.tip, div.warning { margin: 2em ; border: medium outset ; padding: 1em } div.admonition p.admonition-title, div.hint p.admonition-title, div.important p.admonition-title, div.note p.admonition-title, div.tip p.admonition-title { font-weight: bold ; font-family: sans-serif } div.attention p.admonition-title, div.caution p.admonition-title, div.danger p.admonition-title, div.error p.admonition-title, div.warning p.admonition-title { color: red ; font-weight: bold ; font-family: sans-serif } /* Uncomment (and remove this text!) to get reduced vertical space in compound paragraphs. div.compound .compound-first, div.compound .compound-middle { margin-bottom: 0.5em } div.compound .compound-last, div.compound .compound-middle { margin-top: 0.5em } */ div.dedication { margin: 2em 5em ; text-align: center ; font-style: italic } div.dedication p.topic-title { font-weight: bold ; font-style: normal } div.figure { margin-left: 2em ; margin-right: 2em } div.footer, div.header { clear: both; font-size: smaller } div.line-block { display: block ; margin-top: 1em ; margin-bottom: 1em } div.line-block div.line-block { margin-top: 0 ; margin-bottom: 0 ; margin-left: 1.5em } div.sidebar { margin-left: 1em ; border: medium outset ; padding: 1em ; background-color: #ffffee ; width: 40% ; float: right ; clear: right } div.sidebar p.rubric { font-family: sans-serif ; font-size: medium } div.system-messages { margin: 5em } div.system-messages h1 { color: red } div.system-message { border: medium outset ; padding: 1em } div.system-message p.system-message-title { color: red ; font-weight: bold } div.topic { margin: 2em } h1.section-subtitle, h2.section-subtitle, h3.section-subtitle, h4.section-subtitle, h5.section-subtitle, h6.section-subtitle { margin-top: 0.4em } h1.title { text-align: center } h2.subtitle { text-align: center } hr.docutils { width: 75% } img.align-left { clear: left } img.align-right { clear: right } ol.simple, ul.simple { margin-bottom: 1em } ol.arabic { list-style: decimal } ol.loweralpha { list-style: lower-alpha } ol.upperalpha { list-style: upper-alpha } ol.lowerroman { list-style: lower-roman } ol.upperroman { list-style: upper-roman } p.attribution { text-align: right ; margin-left: 50% } p.caption { font-style: italic } p.credits { font-style: italic ; font-size: smaller } p.label { white-space: nowrap } p.rubric { font-weight: bold ; font-size: larger ; color: maroon ; text-align: center } p.sidebar-title { font-family: sans-serif ; font-weight: bold ; font-size: larger } p.sidebar-subtitle { font-family: sans-serif ; font-weight: bold } p.topic-title { font-weight: bold } pre.address { margin-bottom: 0 ; margin-top: 0 ; font-family: serif ; font-size: 100% } pre.literal-block, pre.doctest-block { margin-left: 2em ; margin-right: 2em ; background-color: #eeeeee } span.classifier { font-family: sans-serif ; font-style: oblique } span.classifier-delimiter { font-family: sans-serif ; font-weight: bold } span.interpreted { font-family: sans-serif } span.option { white-space: nowrap } span.pre { white-space: pre } span.problematic { color: red } span.section-subtitle { /* font-size relative to parent (h1..h6 element) */ font-size: 80% } table.citation { border-left: solid 1px gray; margin-left: 1px } table.docinfo { margin: 2em 4em } table.docutils { margin-top: 0.5em ; margin-bottom: 0.5em } table.footnote { border-left: solid 1px black; margin-left: 1px } table.docutils td, table.docutils th, table.docinfo td, table.docinfo th { padding-left: 0.5em ; padding-right: 0.5em ; vertical-align: top } table.docutils th.field-name, table.docinfo th.docinfo-name { font-weight: bold ; text-align: left ; white-space: nowrap ; padding-left: 0 } h1 tt.docutils, h2 tt.docutils, h3 tt.docutils, h4 tt.docutils, h5 tt.docutils, h6 tt.docutils { font-size: 100% } tt.docutils { background-color: #eeeeee } ul.auto-toc { list-style-type: none } </style> </head> <body> <div class="document" id="using-the-pyparsing-module"> <h1 class="title">Using the pyparsing module</h1> <table class="docinfo" frame="void" rules="none"> <col class="docinfo-name" /> <col class="docinfo-content" /> <tbody valign="top"> <tr><th class="docinfo-name">Author:</th> <td>Paul McGuire</td></tr> <tr><th class="docinfo-name">Address:</th> <td><pre class="address"> <a class="first last reference" href="mailto:ptmcg@users.sourceforge.net">ptmcg@users.sourceforge.net</a> </pre> </td></tr> <tr><th class="docinfo-name">Revision:</th> <td>1.4.2</td></tr> <tr><th class="docinfo-name">Date:</th> <td>April, 2006</td></tr> <tr><th class="docinfo-name">Copyright:</th> <td>Copyright © 2003-2006 Paul McGuire.</td></tr> </tbody> </table> <table class="docutils field-list" frame="void" rules="none"> <col class="field-name" /> <col class="field-body" /> <tbody valign="top"> <tr class="field"><th class="field-name">abstract:</th><td class="field-body">This document provides how-to instructions for the pyparsing library, an easy-to-use Python module for constructing and executing basic text parsers. The pyparsing module is useful for evaluating user-definable expressions, processing custom application language commands, or extracting data from formatted reports.</td> </tr> </tbody> </table> <div class="contents topic"> <p class="topic-title first"><a id="contents" name="contents">Contents</a></p> <ul class="auto-toc simple"> <li><a class="reference" href="#steps-to-follow" id="id1" name="id1">1 Steps to follow</a><ul class="auto-toc"> <li><a class="reference" href="#hello-world" id="id2" name="id2">1.1 Hello, World!</a></li> <li><a class="reference" href="#usage-notes" id="id3" name="id3">1.2 Usage notes</a></li> </ul> </li> <li><a class="reference" href="#classes" id="id4" name="id4">2 Classes</a><ul class="auto-toc"> <li><a class="reference" href="#classes-in-the-pyparsing-module" id="id5" name="id5">2.1 Classes in the pyparsing module</a></li> <li><a class="reference" href="#basic-parserelement-subclasses" id="id6" name="id6">2.2 Basic ParserElement subclasses</a></li> <li><a class="reference" href="#expression-subclasses" id="id7" name="id7">2.3 Expression subclasses</a></li> <li><a class="reference" href="#positional-subclasses" id="id8" name="id8">2.4 Positional subclasses</a></li> <li><a class="reference" href="#converter-subclasses" id="id9" name="id9">2.5 Converter subclasses</a></li> <li><a class="reference" href="#special-subclasses" id="id10" name="id10">2.6 Special subclasses</a></li> <li><a class="reference" href="#exception-classes-and-troubleshooting" id="id11" name="id11">2.7 Exception classes and Troubleshooting</a></li> </ul> </li> <li><a class="reference" href="#miscellaneous-attributes-and-methods" id="id12" name="id12">3 Miscellaneous attributes and methods</a><ul class="auto-toc"> <li><a class="reference" href="#helper-methods" id="id13" name="id13">3.1 Helper methods</a></li> <li><a class="reference" href="#helper-parse-actions" id="id14" name="id14">3.2 Helper parse actions</a></li> <li><a class="reference" href="#common-string-and-token-constants" id="id15" name="id15">3.3 Common string and token constants</a></li> </ul> </li> </ul> </div> <div class="section"> <h1><a class="toc-backref" href="#id1" id="steps-to-follow" name="steps-to-follow">1 Steps to follow</a></h1> <p>To parse an incoming data string, the client code must follow these steps:</p> <ol class="arabic simple"> <li>First define the tokens and patterns to be matched, and assign this to a program variable. Optional results names or parsing actions can also be defined at this time.</li> <li>Call <tt class="docutils literal"><span class="pre">parseString()</span></tt> or <tt class="docutils literal"><span class="pre">scanString()</span></tt> on this variable, passing in the string to be parsed. During the matching process, whitespace between tokens is skipped by default (although this can be changed). When token matches occur, any defined parse action methods are called.</li> <li>Process the parsed results, returned as a list of strings. Matching results may also be accessed as named attributes of the returned results, if names are defined in the definition of the token pattern, using <tt class="docutils literal"><span class="pre">setResultsName()</span></tt>.</li> </ol> <div class="section"> <h2><a class="toc-backref" href="#id2" id="hello-world" name="hello-world">1.1 Hello, World!</a></h2> <p>The following complete Python program will parse the greeting "Hello, World!", or any other greeting of the form "<salutation>, <addressee>!":</p> <pre class="literal-block"> from pyparsing import Word, alphas greet = Word( alphas ) + "," + Word( alphas ) + "!" greeting = greet.parseString( "Hello, World!" ) print greeting </pre> <p>The parsed tokens are returned in the following form:</p> <pre class="literal-block"> ['Hello', ',', 'World', '!'] </pre> </div> <div class="section"> <h2><a class="toc-backref" href="#id3" id="usage-notes" name="usage-notes">1.2 Usage notes</a></h2> <ul> <li><p class="first">The pyparsing module can be used to interpret simple command strings or algebraic expressions, or can be used to extract data from text reports with complicated format and structure ("screen or report scraping"). However, it is possible that your defined matching patterns may accept invalid inputs. Use pyparsing to extract data from strings assumed to be well-formatted.</p> </li> <li><p class="first">To keep up the readability of your code, use the <tt class="docutils literal"><span class="pre">+</span></tt>, <tt class="docutils literal"><span class="pre">|</span></tt>, <tt class="docutils literal"><span class="pre">^</span></tt>, and <tt class="docutils literal"><span class="pre">~</span></tt> operators to combine expressions. You can also combine string literals with ParseExpressions - they will be automatically converted to Literal objects. For example:</p> <pre class="literal-block"> integer = Word( nums ) # simple unsigned integer variable = Word( alphas, max=1 ) # single letter variable, such as x, z, m, etc. arithOp = Word( "+-*/", max=1 ) # arithmetic operators equation = variable + "=" + integer + arithOp + integer # will match "x=2+2", etc. </pre> <p>In the definition of <tt class="docutils literal"><span class="pre">equation</span></tt>, the string <tt class="docutils literal"><span class="pre">"="</span></tt> will get added as a <tt class="docutils literal"><span class="pre">Literal("=")</span></tt>, but in a more readable way.</p> </li> <li><p class="first">The pyparsing module's default behavior is to ignore whitespace. This is the case for 99% of all parsers ever written. This allows you to write simple, clean, grammars, such as the above <tt class="docutils literal"><span class="pre">equation</span></tt>, without having to clutter it up with extraneous <tt class="docutils literal"><span class="pre">ws</span></tt> markers. The <tt class="docutils literal"><span class="pre">equation</span></tt> grammar will successfully parse all of the following statements:</p> <pre class="literal-block"> x=2+2 x = 2+2 a = 10 * 4 r= 1234/ 100000 </pre> <p>Of course, it is quite simple to extend this example to support more elaborate expressions, with nesting with parentheses, floating point numbers, scientific notation, and named constants (such as <tt class="docutils literal"><span class="pre">e</span></tt> or <tt class="docutils literal"><span class="pre">pi</span></tt>). See <tt class="docutils literal"><span class="pre">fourFn.py</span></tt>, included in the examples directory.</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">MatchFirst</span></tt> expressions are matched left-to-right, and the first match found will skip all later expressions within, so be sure to define less-specific patterns after more-specific patterns. If you are not sure which expressions are most specific, use Or expressions (defined using the <tt class="docutils literal"><span class="pre">^</span></tt> operator) - they will always match the longest expression, although they are more compute-intensive.</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">Or</span></tt> expressions will evaluate all of the specified subexpressions to determine which is the "best" match, that is, which matches the longest string in the input data. In case of a tie, the left-most expression in the <tt class="docutils literal"><span class="pre">Or</span></tt> list will win.</p> </li> <li><p class="first">If parsing the contents of an entire file, pass it to the parseFile method using:</p> <pre class="literal-block"> expr.parseFile( sourceFile ) </pre> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">ParseExceptions</span></tt> will report the location where an expected token or expression failed to match. For example, if we tried to use our "Hello, World!" parser to parse "Hello World!" (leaving out the separating comma), we would get an exception, with the message:</p> <pre class="literal-block"> pyparsing.ParseException: Expected "," (6), (1,7) </pre> <p>In the case of complex expressions, the reported location may not be exactly where you would expect. See more information under <a class="reference" href="#parseexception">ParseException</a> .</p> </li> <li><p class="first">Use the <tt class="docutils literal"><span class="pre">Group</span></tt> class to enclose logical groups of tokens within a sublist. This will help organize your results into more hierarchical form (the default behavior is to return matching tokens as a flat list of matching input strings).</p> </li> <li><p class="first">Punctuation may be significant for matching, but is rarely of much interest in the parsed results. Use the <tt class="docutils literal"><span class="pre">suppress()</span></tt> method to keep these tokens from cluttering up your returned lists of tokens. For example, <tt class="docutils literal"><span class="pre">delimitedList()</span></tt> matches a succession of one or more expressions, separated by delimiters (commas by default), but only returns a list of the actual expressions - the delimiters are used for parsing, but are suppressed from the returned output.</p> </li> <li><p class="first">Parse actions can be used to convert values from strings to other data types (ints, floats, booleans, etc.).</p> </li> <li><p class="first">Be careful when defining parse actions that modify global variables or data structures (as in <tt class="docutils literal"><span class="pre">fourFn.py</span></tt>), especially for low level tokens or expressions that may occur within an <tt class="docutils literal"><span class="pre">And</span></tt> expression; an early element of an <tt class="docutils literal"><span class="pre">And</span></tt> may match, but the overall expression may fail.</p> </li> <li><p class="first">Performance of pyparsing may be slow for complex grammars and/or large input strings. The <a class="reference" href="http://psyco.sourceforge.net/">psyco</a> package can be used to improve the speed of the pyparsing module with no changes to grammar or program logic - observed improvments have been in the 20-50% range.</p> </li> </ul> </div> </div> <div class="section"> <h1><a class="toc-backref" href="#id4" id="classes" name="classes">2 Classes</a></h1> <div class="section"> <h2><a class="toc-backref" href="#id5" id="classes-in-the-pyparsing-module" name="classes-in-the-pyparsing-module">2.1 Classes in the pyparsing module</a></h2> <p><tt class="docutils literal"><span class="pre">ParserElement</span></tt> - abstract base class for all pyparsing classes; methods for code to use are:</p> <ul> <li><p class="first"><tt class="docutils literal"><span class="pre">parseString(</span> <span class="pre">sourceString</span> <span class="pre">)</span></tt> - only called once, on the overall matching pattern; returns a <a class="reference" href="#parseresults">ParseResults</a> object that makes the matched tokens available as a list, and optionally as a dictionary, or as an object with named attributes</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">parseFile(</span> <span class="pre">sourceFile</span> <span class="pre">)</span></tt> - a convenience function, that accepts an input file object or filename. The file contents are passed as a string to <tt class="docutils literal"><span class="pre">parseString()</span></tt>.</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">scanString(</span> <span class="pre">sourceString</span> <span class="pre">)</span></tt> - generator function, used to find and extract matching text in the given source string; for each matched text, returns a tuple of:</p> <ul class="simple"> <li>matched tokens (packaged as a <a class="reference" href="#parseresults">ParseResults</a> object)</li> <li>start location of the matched text in the given source string</li> <li>end location in the given source string</li> </ul> <p><tt class="docutils literal"><span class="pre">scanString</span></tt> allows you to scan through the input source string for random matches, instead of exhaustively defining the grammar for the entire source text (as would be required with <tt class="docutils literal"><span class="pre">parseString</span></tt>).</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">transformString(</span> <span class="pre">sourceString</span> <span class="pre">)</span></tt> - convenience wrapper function for <tt class="docutils literal"><span class="pre">scanString</span></tt>, to process the input source string, and replace matching text with the tokens returned from parse actions defined in the grammar (see <a class="reference" href="#setparseaction">setParseAction</a>).</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">searchString(</span> <span class="pre">sourceString</span> <span class="pre">)</span></tt> - another convenience wrapper function for <tt class="docutils literal"><span class="pre">scanString</span></tt>, returns a list of the matching tokens returned from each call to <tt class="docutils literal"><span class="pre">scanString</span></tt>.</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">setName(</span> <span class="pre">name</span> <span class="pre">)</span></tt> - associate a short descriptive name for this element, useful in displaying exceptions and trace information</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">setResultsName(</span> <span class="pre">string,</span> <span class="pre">listAllMatches=False</span> <span class="pre">)</span></tt> - name to be given to tokens matching the element; if multiple tokens within a repetition group (such as <tt class="docutils literal"><span class="pre">ZeroOrMore</span></tt> or <tt class="docutils literal"><span class="pre">delimitedList</span></tt>) the default is to return only the last matching token - if listAllMatches is set to True, then a list of matching tokens is returned. Note: <tt class="docutils literal"><span class="pre">setResultsName</span></tt> returns a <em>copy</em> of the element so that a single basic element can be referenced multiple times and given different names within a complex grammar.</p> </li> </ul> <ul id="setparseaction"> <li><p class="first"><tt class="docutils literal"><span class="pre">setParseAction(</span> <span class="pre">*fn</span> <span class="pre">)</span></tt> - specify one or more functions to call after successful matching of the element; each function is defined as <tt class="docutils literal"><span class="pre">fn(</span> <span class="pre">s,</span> <span class="pre">loc,</span> <span class="pre">toks</span> <span class="pre">)</span></tt>, where:</p> <ul class="simple"> <li><tt class="docutils literal"><span class="pre">s</span></tt> is the original parse string</li> <li><tt class="docutils literal"><span class="pre">loc</span></tt> is the location in the string where matching started</li> <li><tt class="docutils literal"><span class="pre">toks</span></tt> is the list of the matched tokens, packaged as a <a class="reference" href="#parseresults">ParseResults</a> object</li> </ul> <p>Multiple functions can be attached to a ParserElement by specifying multiple arguments to setParseAction, or by calling setParseAction multiple times.</p> <p>Each parse action function can return a modified <tt class="docutils literal"><span class="pre">toks</span></tt> list, to perform conversion, or string modifications. For brevity, <tt class="docutils literal"><span class="pre">fn</span></tt> may also be a lambda - here is an example of using a parse action to convert matched integer tokens from strings to integers:</p> <pre class="literal-block"> intNumber = Word(nums).setParseAction( lambda s,l,t: [ int(t[0]) ] ) </pre> <p>If <tt class="docutils literal"><span class="pre">fn</span></tt> does not modify the <tt class="docutils literal"><span class="pre">toks</span></tt> list, it does not need to return anything at all.</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">copy()</span></tt> - returns a copy of a ParserElement; can be used to use the same parse expression in different places in a grammar, with different parse actions attached to each</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">leaveWhiteSpace()</span></tt> - change default behavior of skipping whitespace before starting matching (mostly used internally to the pyparsing module, rarely used by client code)</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">setWhitespaceChars(</span> <span class="pre">chars</span> <span class="pre">)</span></tt> - define the set of chars to be ignored as whitespace before trying to match a specific ParserElement, in place of the default set of whitespace (space, tab, newline, and return)</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">setDefaultWhitespaceChars(</span> <span class="pre">chars</span> <span class="pre">)</span></tt> - class-level method to override the default set of whitespace chars for all subsequently created ParserElements (including copies); useful when defining grammars that treat one or more of the default whitespace characters as significant (such as a line-sensitive grammar, to omit newline from the list of ignorable whitespace)</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">suppress()</span></tt> - convenience function to suppress the output of the given element, instead of wrapping it with a Suppress object.</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">ignore(</span> <span class="pre">expr</span> <span class="pre">)</span></tt> - function to specify parse expression to be ignored while matching defined patterns; can be called repeatedly to specify multiple expressions; useful to specify patterns of comment syntax, for example</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">setDebug(</span> <span class="pre">dbgFlag=True</span> <span class="pre">)</span></tt> - function to enable/disable tracing output when trying to match this element</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">validate()</span></tt> - function to verify that the defined grammar does not contain infinitely recursive constructs</p> </li> </ul> <ul class="simple" id="parsewithtabs"> <li><tt class="docutils literal"><span class="pre">parseWithTabs()</span></tt> - function to override default behavior of converting tabs to spaces before parsing the input string; rarely used, except when specifying whitespace-significant grammars using the <a class="reference" href="#white">White</a> class.</li> <li><tt class="docutils literal"><span class="pre">enablePackrat()</span></tt> - a class-level static method to enable a memoizing performance enhancement, known as "packrat parsing". packrat parsing is disabled by default, since it may conflict with some user programs that use parse actions. To activate the packrat feature, your program must call the class method ParserElement.enablePackrat(). If your program uses psyco to "compile as you go", you must call enablePackrat before calling psyco.full(). If you do not do this, Python will crash. For best results, call enablePackrat() immediately after importing pyparsing.</li> </ul> </div> <div class="section"> <h2><a class="toc-backref" href="#id6" id="basic-parserelement-subclasses" name="basic-parserelement-subclasses">2.2 Basic ParserElement subclasses</a></h2> <ul class="simple"> <li><tt class="docutils literal"><span class="pre">Literal</span></tt> - construct with a string to be matched exactly</li> <li><tt class="docutils literal"><span class="pre">CaselessLiteral</span></tt> - construct with a string to be matched, but without case checking; results are always returned as the defining literal, NOT as they are found in the input string</li> <li><tt class="docutils literal"><span class="pre">Keyword</span></tt> - similar to Literal, but must be immediately followed by whitespace, punctuation, or other non-keyword characters; prevents accidental matching of a non-keyword that happens to begin with a defined keyword</li> <li><tt class="docutils literal"><span class="pre">CaselessKeyword</span></tt> - similar to Keyword, but with caseless matching behavior</li> </ul> <ul id="word"> <li><p class="first"><tt class="docutils literal"><span class="pre">Word</span></tt> - one or more contiguous characters; construct with a string containing the set of allowed initial characters, and an optional second string of allowed body characters; for instance, a common Word construct is to match a code identifier - in C, a valid identifier must start with an alphabetic character or an underscore ('_'), followed by a body that can also include numeric digits. That is, <tt class="docutils literal"><span class="pre">a</span></tt>, <tt class="docutils literal"><span class="pre">i</span></tt>, <tt class="docutils literal"><span class="pre">MAX_LENGTH</span></tt>, <tt class="docutils literal"><span class="pre">_a1</span></tt>, <tt class="docutils literal"><span class="pre">b_109_</span></tt>, and <tt class="docutils literal"><span class="pre">plan9FromOuterSpace</span></tt> are all valid identifiers; <tt class="docutils literal"><span class="pre">9b7z</span></tt>, <tt class="docutils literal"><span class="pre">$a</span></tt>, <tt class="docutils literal"><span class="pre">.section</span></tt>, and <tt class="docutils literal"><span class="pre">0debug</span></tt> are not. To define an identifier using a Word, use either of the following:</p> <pre class="literal-block"> - Word( alphas+"_", alphanums+"_" ) - Word( srange("[a-zA-Z_]"), srange("[a-zA-Z0-9_]") ) </pre> <p>If only one string given, it specifies that the same character set defined for the initial character is used for the word body; for instance, to define an identifier that can only be composed of capital letters and underscores, use:</p> <pre class="literal-block"> - Word( "ABCDEFGHIJKLMNOPQRSTUVWXYZ" ) - Word( srange("[A-Z]") </pre> <p>A Word may also be constructed with any of the following optional parameters:</p> <ul class="simple"> <li>min - indicating a minimum length of matching characters</li> <li>max - indicating a maximum length of matching characters</li> <li>exact - indicating an exact length of matching characters</li> </ul> <p>If exact is specified, it will override any values for min or max.</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">CharsNotIn</span></tt> - similar to <a class="reference" href="#word">Word</a>, but matches characters not in the given constructor string (accepts only one string for both initial and body characters); also supports min, max, and exact optional parameters.</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">Regex</span></tt> - a powerful construct, that accepts a regular expression to be matched at the current parse position; accepts an optional flags parameter, corresponding to the flags parameter in the re.compile method; if the expression includes named sub-fields, they will be represented in the returned <a class="reference" href="#parseresults">ParseResults</a></p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">QuotedString</span></tt> - supports the definition of custom quoted string formats, in addition to pyparsing's built-in dblQuotedString and sglQuotedString. QuotedString allows you to specify the following parameters:</p> <ul class="simple"> <li>quoteChar - string of one or more characters defining the quote delimiting string</li> <li>escChar - character to escape quotes, typically backslash (default=None)</li> <li>escQuote - special quote sequence to escape an embedded quote string (such as SQL's "" to escape an embedded ") (default=None)</li> <li>multiline - boolean indicating whether quotes can span multiple lines (default=False)</li> <li>unquoteResults - boolean indicating whether the matched text should be unquoted (default=True)</li> <li>endQuoteChar - string of one or more characters defining the end of the quote delimited string (default=None => same as quoteChar)</li> </ul> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">SkipTo</span></tt> - skips ahead in the input string, accepting any characters up to the specified pattern; may be constructed with the following optional parameters:</p> <ul class="simple"> <li>include - if set to true, also consumes the match expression (default is false)</li> <li>ignore - allows the user to specify patterns to not be matched, to prevent false matches</li> </ul> </li> </ul> <ul class="simple" id="white"> <li><tt class="docutils literal"><span class="pre">White</span></tt> - also similar to <a class="reference" href="#word">Word</a>, but matches whitespace characters. Not usually needed, as whitespace is implicitly ignored by pyparsing. However, some grammars are whitespace-sensitive, such as those that use leading tabs or spaces to indicating grouping or hierarchy. (If matching on tab characters, be sure to call <a class="reference" href="#parsewithtabs">parseWithTabs</a> on the top-level parse element.)</li> <li><tt class="docutils literal"><span class="pre">Empty</span></tt> - a null expression, requiring no characters - will always match; useful for debugging and for specialized grammars</li> <li><tt class="docutils literal"><span class="pre">NoMatch</span></tt> - opposite of Empty, will never match; useful for debugging and for specialized grammars</li> </ul> </div> <div class="section"> <h2><a class="toc-backref" href="#id7" id="expression-subclasses" name="expression-subclasses">2.3 Expression subclasses</a></h2> <ul class="simple"> <li><tt class="docutils literal"><span class="pre">And</span></tt> - construct with a list of ParserElements, all of which must match for And to match; can also be created using the '+' operator</li> <li><tt class="docutils literal"><span class="pre">Or</span></tt> - construct with a list of ParserElements, any of which must match for Or to match; if more than one expression matches, the expression that makes the longest match will be used; can also be created using the '^' operator</li> <li><tt class="docutils literal"><span class="pre">MatchFirst</span></tt> - construct with a list of ParserElements, any of which must match for MatchFirst to match; matching is done left-to-right, taking the first expression that matches; can also be created using the '|' operator</li> <li><tt class="docutils literal"><span class="pre">Each</span></tt> - similar to And, in that all of the provided expressions must match; however, Each permits matching to be done in any order; can also be created using the '&' operator</li> <li><tt class="docutils literal"><span class="pre">Optional</span></tt> - construct with a ParserElement, but this element is not required to match; can be constructed with an optional <tt class="docutils literal"><span class="pre">default</span></tt> argument, containing a default string or object to be supplied if the given optional parse element is not found in the input string; parse action will only be called if a match is found, or if a default is specified</li> <li><tt class="docutils literal"><span class="pre">ZeroOrMore</span></tt> - similar to Optional, but can be repeated</li> <li><tt class="docutils literal"><span class="pre">OneOrMore</span></tt> - similar to ZeroOrMore, but at least one match must be present</li> <li><tt class="docutils literal"><span class="pre">FollowedBy</span></tt> - a lookahead expression, requires matching of the given expressions, but does not advance the parsing position within the input string</li> <li><tt class="docutils literal"><span class="pre">NotAny</span></tt> - a negative lookahead expression, prevents matching of named expressions, does not advance the parsing position within the input string; can also be created using the unary '~' operator</li> </ul> </div> <div class="section"> <h2><a class="toc-backref" href="#id8" id="positional-subclasses" name="positional-subclasses">2.4 Positional subclasses</a></h2> <ul class="simple"> <li><tt class="docutils literal"><span class="pre">StringStart</span></tt> - matches beginning of the text</li> <li><tt class="docutils literal"><span class="pre">StringEnd</span></tt> - matches the end of the text</li> <li><tt class="docutils literal"><span class="pre">LineStart</span></tt> - matches beginning of a line (lines delimited by <tt class="docutils literal"><span class="pre">\n</span></tt> characters)</li> <li><tt class="docutils literal"><span class="pre">LineEnd</span></tt> - matches the end of a line</li> </ul> </div> <div class="section"> <h2><a class="toc-backref" href="#id9" id="converter-subclasses" name="converter-subclasses">2.5 Converter subclasses</a></h2> <ul class="simple"> <li><tt class="docutils literal"><span class="pre">Upcase</span></tt> - converts matched tokens to uppercase</li> <li><tt class="docutils literal"><span class="pre">Combine</span></tt> - joins all matched tokens into a single string, using specified joinString (default <tt class="docutils literal"><span class="pre">joinString=""</span></tt>); expects all matching tokens to be adjacent, with no intervening whitespace (can be overridden by specifying <tt class="docutils literal"><span class="pre">adjacent=False</span></tt> in constructor)</li> <li><tt class="docutils literal"><span class="pre">Suppress</span></tt> - clears matched tokens; useful to keep returned results from being cluttered with required but uninteresting tokens (such as list delimiters)</li> </ul> </div> <div class="section"> <h2><a class="toc-backref" href="#id10" id="special-subclasses" name="special-subclasses">2.6 Special subclasses</a></h2> <ul class="simple"> <li><tt class="docutils literal"><span class="pre">Group</span></tt> - causes the matched tokens to be enclosed in a list; useful in repeated elements like <tt class="docutils literal"><span class="pre">ZeroOrMore</span></tt> and <tt class="docutils literal"><span class="pre">OneOrMore</span></tt> to break up matched tokens into groups for each repeated pattern</li> <li><tt class="docutils literal"><span class="pre">Dict</span></tt> - like <tt class="docutils literal"><span class="pre">Group</span></tt>, but also constructs a dictionary, using the [0]'th elements of all enclosed token lists as the keys, and each token list as the value</li> <li><tt class="docutils literal"><span class="pre">SkipTo</span></tt> - catch-all matching expression that accepts all characters up until the given pattern is found to match; useful for specifying incomplete grammars</li> <li><tt class="docutils literal"><span class="pre">Forward</span></tt> - placeholder token used to define recursive token patterns; when defining the actual expression later in the program, insert it into the <tt class="docutils literal"><span class="pre">Forward</span></tt> object using the <tt class="docutils literal"><span class="pre"><<</span></tt> operator (see <tt class="docutils literal"><span class="pre">fourFn.py</span></tt> for an example).</li> </ul> <ul id="parseresults"> <li><p class="first"><tt class="docutils literal"><span class="pre">ParseResults</span></tt> - class used to contain and manage the lists of tokens created from parsing the input using the user-defined parse expression. ParseResults can be accessed in a number of ways:</p> <ul class="simple"> <li>as a list<ul> <li>total list of elements can be found using len()</li> <li>individual elements can be found using [0], [1], [-1], etc.</li> <li>elements can be deleted using del</li> </ul> </li> <li>as a dictionary<ul> <li>if <tt class="docutils literal"><span class="pre">setResultsName()</span></tt> is used to name elements within the overall parse expression, then these fields can be referenced as dictionary elements or as attributes</li> <li>the Dict class generates dictionary entries using the data of the input text - in addition to ParseResults listed as <tt class="docutils literal"><span class="pre">[</span> <span class="pre">[</span> <span class="pre">a1,</span> <span class="pre">b1,</span> <span class="pre">c1,</span> <span class="pre">...],</span> <span class="pre">[</span> <span class="pre">a2,</span> <span class="pre">b2,</span> <span class="pre">c2,</span> <span class="pre">...]</span> <span class="pre">]</span></tt> it also acts as a dictionary with entries defined as <tt class="docutils literal"><span class="pre">{</span> <span class="pre">a1</span> <span class="pre">:</span> <span class="pre">[</span> <span class="pre">b1,</span> <span class="pre">c1,</span> <span class="pre">...</span> <span class="pre">]</span> <span class="pre">},</span> <span class="pre">{</span> <span class="pre">a2</span> <span class="pre">:</span> <span class="pre">[</span> <span class="pre">b2,</span> <span class="pre">c2,</span> <span class="pre">...</span> <span class="pre">]</span> <span class="pre">}</span></tt>; this is especially useful when processing tabular data where the first column contains a key value for that line of data</li> <li>list elements that are deleted using <tt class="docutils literal"><span class="pre">del</span></tt> will still be accessible by their dictionary keys</li> <li>supports <tt class="docutils literal"><span class="pre">items()</span></tt> and <tt class="docutils literal"><span class="pre">keys()</span></tt> methods, similar to a dictionary</li> </ul> </li> <li>as a nested list<ul> <li>results returned from the Group class are encapsulated within their own list structure, so that the tokens can be handled as a hierarchical tree</li> </ul> </li> </ul> <p>ParseResults can also be converted to an ordinary list of strings by calling <tt class="docutils literal"><span class="pre">asList()</span></tt>. Note that this will strip the results of any field names that have been defined for any embedded parse elements. (The <tt class="docutils literal"><span class="pre">pprint</span></tt> module is especially good at printing out the nested contents given by <tt class="docutils literal"><span class="pre">asList()</span></tt>.)</p> <p>Finally, ParseResults can be converted to an XML string by calling <tt class="docutils literal"><span class="pre">asXML()</span></tt>. Where possible, results will be tagged using the results names defined for the respective ParseExpressions. <tt class="docutils literal"><span class="pre">asXML()</span></tt> takes two optional arguments:</p> <ul class="simple"> <li>doctagname - for ParseResults that do not have a defined name, this argument will wrap the resulting XML in a set of opening and closing tags <tt class="docutils literal"><span class="pre"><doctagname></span></tt> and <tt class="docutils literal"><span class="pre"></doctagname></span></tt>.</li> <li>namedItemsOnly (default=False) - flag to indicate if the generated XML should skip items that do not have defined names. If a nested group item is named, then all embedded items will be included, whether they have names or not.</li> </ul> </li> </ul> </div> <div class="section"> <h2><a class="toc-backref" href="#id11" id="exception-classes-and-troubleshooting" name="exception-classes-and-troubleshooting">2.7 Exception classes and Troubleshooting</a></h2> <ul id="parseexception"> <li><p class="first"><tt class="docutils literal"><span class="pre">ParseException</span></tt> - exception returned when a grammar parse fails; ParseExceptions have attributes loc, msg, line, lineno, and column; to view the text line and location where the reported ParseException occurs, use:</p> <pre class="literal-block"> except ParseException, err: print err.line print " "*(err.column-1) + "^" print err </pre> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">RecursiveGrammarException</span></tt> - exception returned by <tt class="docutils literal"><span class="pre">validate()</span></tt> if the grammar contains a recursive infinite loop, such as:</p> <pre class="literal-block"> badGrammar = Forward() goodToken = Literal("A") badGrammar << Optional(goodToken) + badGrammar </pre> </li> </ul> <p>You can also get some insights into the parsing logic using diagnostic parse actions, and setDebug(), or test the matching of expression fragments by testing them using scanString().</p> </div> </div> <div class="section"> <h1><a class="toc-backref" href="#id12" id="miscellaneous-attributes-and-methods" name="miscellaneous-attributes-and-methods">3 Miscellaneous attributes and methods</a></h1> <div class="section"> <h2><a class="toc-backref" href="#id13" id="helper-methods" name="helper-methods">3.1 Helper methods</a></h2> <ul class="simple"> <li><tt class="docutils literal"><span class="pre">delimitedList(</span> <span class="pre">expr,</span> <span class="pre">delim=',')</span></tt> - convenience function for matching one or more occurrences of expr, separated by delim. By default, the delimiters are suppressed, so the returned results contain only the separate list elements. Can optionally specify <tt class="docutils literal"><span class="pre">combine=True</span></tt>, indicating that the expressions and delimiters should be returned as one combined value (useful for scoped variables, such as <tt class="docutils literal"><span class="pre">a.b.c</span></tt>, or <tt class="docutils literal"><span class="pre">a::b::c</span></tt>, or paths such as <tt class="docutils literal"><span class="pre">a/b/c</span></tt>).</li> <li><tt class="docutils literal"><span class="pre">countedArray(</span> <span class="pre">expr</span> <span class="pre">)</span></tt> - convenience function for a pattern where an list of instances of the given expression are preceded by an integer giving the count of elements in the list. Returns an expression that parses the leading integer, reads exactly that many expressions, and returns the array of expressions in the parse results - the leading integer is suppressed from the results (although it is easily reconstructed by using len on the returned array).</li> <li><tt class="docutils literal"><span class="pre">oneOf(</span> <span class="pre">string,</span> <span class="pre">caseless=False</span> <span class="pre">)</span></tt> - convenience function for quickly declaring an alternative set of <tt class="docutils literal"><span class="pre">Literal</span></tt> tokens, by splitting the given string on whitespace boundaries. The tokens are sorted so that longer matches are attempted first; this ensures that a short token does not mask a longer one that starts with the same characters. If <tt class="docutils literal"><span class="pre">caseless=True</span></tt>, will create an alternative set of CaselessLiteral tokens.</li> <li><tt class="docutils literal"><span class="pre">dictOf(</span> <span class="pre">key,</span> <span class="pre">value</span> <span class="pre">)</span></tt> - convenience function for quickly declaring a dictionary pattern of <tt class="docutils literal"><span class="pre">Dict(</span> <span class="pre">ZeroOrMore(</span> <span class="pre">Group(</span> <span class="pre">key</span> <span class="pre">+</span> <span class="pre">value</span> <span class="pre">)</span> <span class="pre">)</span> <span class="pre">)</span></tt>.</li> <li><tt class="docutils literal"><span class="pre">makeHTMLTags(</span> <span class="pre">tagName</span> <span class="pre">)</span></tt> and <tt class="docutils literal"><span class="pre">makeXMLTags(</span> <span class="pre">tagName</span> <span class="pre">)</span></tt> - convenience functions to create definitions of opening and closing tag expressions. Returns a pair of expressions, for the corresponding <tag> and </tag> strings. Includes support for attributes in the opening tag, such as <tag attr1="abc"> - attributes are returned as keyed tokens in the returned ParseResults. makeHTMLTags is less restrictive than makeXMLTags, especially with respect to case sensitivity.</li> <li><tt class="docutils literal"><span class="pre">lineno(</span> <span class="pre">loc,</span> <span class="pre">string</span> <span class="pre">)</span></tt> - function to give the line number of the location within the string; the first line is line 1, newlines start new rows</li> <li><tt class="docutils literal"><span class="pre">col(</span> <span class="pre">loc,</span> <span class="pre">string</span> <span class="pre">)</span></tt> - function to give the column number of the location within the string; the first column is column 1, newlines reset the column number to 1</li> <li><tt class="docutils literal"><span class="pre">line(</span> <span class="pre">loc,</span> <span class="pre">string</span> <span class="pre">)</span></tt> - function to retrieve the line of text representing <tt class="docutils literal"><span class="pre">lineno(</span> <span class="pre">loc,</span> <span class="pre">string</span> <span class="pre">)</span></tt>; useful when printing out diagnostic messages for exceptions</li> <li><tt class="docutils literal"><span class="pre">srange(</span> <span class="pre">rangeSpec</span> <span class="pre">)</span></tt> - function to define a string of characters, given a string of the form used by regexp string ranges, such as <tt class="docutils literal"><span class="pre">"[0-9]"</span></tt> for all numeric digits, <tt class="docutils literal"><span class="pre">"[A-Z_]"</span></tt> for uppercase characters plus underscore, and so on (note that rangeSpec does not include support for generic regular expressions, just string range specs)</li> </ul> </div> <div class="section"> <h2><a class="toc-backref" href="#id14" id="helper-parse-actions" name="helper-parse-actions">3.2 Helper parse actions</a></h2> <ul> <li><p class="first"><tt class="docutils literal"><span class="pre">removeQuotes</span></tt> - removes the first and last characters of a quoted string; useful to remove the delimiting quotes from quoted strings</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">replaceWith(replString)</span></tt> - returns a parse action that simply returns the replString; useful when using transformString, or converting HTML entities, as in:</p> <pre class="literal-block"> nbsp = Literal("&nbsp;").setParseAction( replaceWith("<BLANK>") ) </pre> </li> </ul> </div> <div class="section"> <h2><a class="toc-backref" href="#id15" id="common-string-and-token-constants" name="common-string-and-token-constants">3.3 Common string and token constants</a></h2> <ul> <li><p class="first"><tt class="docutils literal"><span class="pre">alphas</span></tt> - same as <tt class="docutils literal"><span class="pre">string.letters</span></tt></p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">nums</span></tt> - same as <tt class="docutils literal"><span class="pre">string.digits</span></tt></p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">alphanums</span></tt> - a string containing <tt class="docutils literal"><span class="pre">alphas</span> <span class="pre">+</span> <span class="pre">nums</span></tt></p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">alphas8bit</span></tt> - a string containing alphabetic 8-bit characters:</p> <pre class="literal-block"> ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõöøùúûüýþ </pre> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">printables</span></tt> - same as <tt class="docutils literal"><span class="pre">string.printable</span></tt>, minus the space (<tt class="docutils literal"><span class="pre">'</span> <span class="pre">'</span></tt>) character</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">empty</span></tt> - a global <tt class="docutils literal"><span class="pre">Empty()</span></tt>; will always match</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">sglQuotedString</span></tt> - a string of characters enclosed in 's; may include whitespace, but not newlines</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">dblQuotedString</span></tt> - a string of characters enclosed in "s; may include whitespace, but not newlines</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">quotedString</span></tt> - <tt class="docutils literal"><span class="pre">sglQuotedString</span> <span class="pre">|</span> <span class="pre">dblQuotedString</span></tt></p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">cStyleComment</span></tt> - a comment block delimited by <tt class="docutils literal"><span class="pre">'/*'</span></tt> and <tt class="docutils literal"><span class="pre">'*/'</span></tt> sequences; can span multiple lines, but does not support nesting of comments</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">htmlComment</span></tt> - a comment block delimited by <tt class="docutils literal"><span class="pre">'<!--'</span></tt> and <tt class="docutils literal"><span class="pre">'-->'</span></tt> sequences; can span multiple lines, but does not support nesting of comments</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">commaSeparatedList</span></tt> - similar to <tt class="docutils literal"><span class="pre">delimitedList</span></tt>, except that the list expressions can be any text value, or a quoted string; quoted strings can safely include commas without incorrectly breaking the string into two tokens</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">restOfLine</span></tt> - all remaining printable characters up to but not including the next newline</p> </li> </ul> </div> </div> </div> </body> </html>