<?xml version='1.0'?>
<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.3//EN"
"http://www.oasis-open.org/docbook/xml/4.3/docbookx.dtd"
>
<chapter id="development">
	<title>Using the development version</title>
	<section id="tracing">
		<title>Tracing the execution</title>
		<para>
			The generic flags to add more information to the stack traces or to the messages is <emphasis>-v</emphasis> (verbosity), it is used to display the command-lines executed during a build:
			<programlisting language="shell">
$ waf -v
			</programlisting>
			To display all the traces (useful for bug reports), use the following flag:
			<programlisting language="shell">
$ waf -vvv
			</programlisting>
			Debugging information can be filtered easily with the flag <emphasis>zones</emphasis>:
			<programlisting language="shell">
$ waf --zones=action
			</programlisting>
			Tracing zones must be comma-separated, for example:
			<programlisting language="shell">
$ waf --zones=action,envhash,task_gen
			</programlisting>
		</para>
		<para>
			The Waf module <emphasis>Logs</emphasis> replaces the Python module logging. In the source code, traces are provided by using the <emphasis>debug</emphasis> function, they must obey the format "zone: message" like in the following:
			<programlisting language="python">
Logs.debug("task: task %r must run as it was never run before or the task code changed" % self)
			</programlisting>
			The following zones are used in Waf:
<table id="debugging_zones">
<title>Debugging zones</title>

<tgroup cols='2' align='left' colsep='1' rowsep='1'>

<colspec colname='c1'/>
<colspec colname='c2'/>

<thead>
<row>
  <entry>Zone</entry>
  <entry>Meaning</entry>
</row>
</thead>

<tbody>

<row>
  <entry>runner</entry>
  <entry>command-lines executed (enabled when -v is provided without debugging zones)</entry>
</row>

<row>
  <entry>task_gen</entry>
  <entry>task creation (from task generators)</entry>
</row>

<row>
  <entry>action</entry>
  <entry>functions to execute for building the targets</entry>
</row>

<row>
  <entry>env</entry>
  <entry>environment contents</entry>
</row>

<row>
  <entry>envhash</entry>
  <entry>hashes of the environment objects - helps seeing what changes</entry>
</row>

<row>
  <entry>build</entry>
  <entry>build context operations such as filesystem access</entry>
</row>

<row>
  <entry>deps</entry>
  <entry>implicit dependencies found (task scanners)</entry>
</row>

</tbody>
</tgroup>
</table>

		</para>
	</section>

	<section id="benchmarking">
		<title>Benchmarking</title>
		<para>
			The script <filename>utils/genbench.py</filename> generates a simple benchmark for Waf. The habitual use is the following:
			<programlisting language="shell">
utils/genbench.py /tmp/build 50 100 15 5
cd /tmp/build
waf configure
waf -p -j2
			</programlisting>
			The project created contains 50 libraries with 100 classes for each, each source file having 15 include headers pointing to the same library and 5 headers pointing to the headers of other libraries in the project.
		</para>

		<para>
			The time taken to create the tasks and to resolve the dependencies can be obtained by injecting code to disable the actual compilation, for example:
			<programlisting language="python">
def build(bld):
	import Task
	def touch_func(task):
		for x in task.outputs:
			open(x.abspath(task.env), 'w').close()
	for x in Task.TaskBase.classes.keys():
		cls = Task.TaskBase.classes[x]
		cls.func = touch_func
		cls.color = 'CYAN'
			</programlisting>
		</para>
	</section>

	<section id="profiling">
		<title>Profiling</title>
		<para>
			Profiling information is obtained by calling the module cProfile and by injecting specific code. The two most interesting methods to profile are <emphasis>flush</emphasis> and <emphasis>compile</emphasis>. The most important number from the profiling information is the amount of function calls, and reducing it results in noticeable speedups. Here is an example on the method compile:
			<programlisting language="python">
import Build
def ncomp(self):
	import cProfile, pstats
	cProfile.run('import Build; Build.bld.rep_compile()', 'profi.txt')
	p = pstats.Stats('profi.txt')
	p.sort_stats('time').print_stats(150)

Build.bld.rep_compile = Build.bld.compile
Build.bld.compile = ncomp
			</programlisting>
			Here the output obtained on a benchmark build created as explained in the previous section:
			<programlisting language="shell">
Sun Nov  9 19:22:59 2008    prof.txt

         959695 function calls (917845 primitive calls) in 3.195 CPU seconds

   Ordered by: internal time
   List reduced from 187 to 10 due to restriction 10

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.531    0.531    0.531    0.531 {cPickle.dump}
73950/42150    0.223    0.000    0.276    0.000 Environment.py:66(__getitem__)
     5000    0.215    0.000    0.321    0.000 Task.py:770(compute_sig_implicit_deps)
   204434    0.180    0.000    0.180    0.000 {method 'update' of '_hashlib.HASH' objects}
     5050    0.170    0.000    0.551    0.000 Task.py:706(sig_vars)
    10000    0.144    0.000    0.301    0.000 Utils.py:98(h_file)
25255/15205    0.113    0.000    0.152    0.000 Node.py:329(abspath)
    15050    0.107    0.000    0.342    0.000 Task.py:481(unique_id)
    15301    0.102    0.000    0.102    0.000 Environment.py:50(variant)
   132062    0.097    0.000    0.097    0.000 {method 'get' of 'dict' objects}
			</programlisting>

			From the profile information, it appears that the hot spots are, in order:
			<itemizedlist>
				<listitem>The persistence implemented by the cPickle module (the cache file to serialize takes about 3Mb)</listitem>
				<listitem>Accessing configuration data from the Environment instances</listitem>
				<listitem>Computing implicit dependencies (the test project contains lots of interleaved dependencies)</listitem>
			</itemizedlist>
		In practice, the time taken by these methods is not significant enough to justify code changes. Also, profiling must be carried out on builds which last at least several seconds.
		</para>
	</section>

	<section id="parallel_tracing">
		<title>Tracing parallel task execution</title>
		<para>
			A special Waf tool named <emphasis>ParallelDebug</emphasis> is used to inject code in Waf modules and obtain information on the execution. This module is provided in the folder <filename>playground</filename> and must be imported in one's project before use:
			<programlisting language="python">
def configure(conf):
	conf.check_tool('ParallelDebug', tooldir='.')
			</programlisting>
			After executing a full build, a trace of the execution is output in <filename>/tmp/test.dat</filename>; it may be processed by other applications such as Gnuplot:
			<programlisting language="shell">
#! /usr/bin/gnuplot -persist
set terminal X11
set ylabel "Amount of jobs running in parallel"
set xlabel "Time in seconds"
set title "Amount of jobs running in parallel (waf -j3)"
unset label
set yrange [-0.1:4]
set ytic 1
plot 'test.dat' using 1:3 with lines title "" lt 3
			</programlisting>
		</para>
	</section>

	<section id="download">
		<title>Obtaining the latest source code</title>
		<para>Waf is hosted on <ulink url="http://code.google.com/p/waf/source">Google code</ulink>, and uses Subversion for source control. To obtain the development copy, use:
		<programlisting language="sh">
svn checkout http://waf.googlecode.com/svn/trunk/ waf-read-only
cd waf-read-only
./waf-light --make-waf
		</programlisting>
		</para>
		<para>
		To avoid regenerating Waf each time, the environment variable WAFDIR can be used to point to the directory containing wafadmin:
		<programlisting language="sh">
export WAFDIR=/path/to/wafadmin
		</programlisting>
		</para>
		<para>
			Although the Waf binary depends on Python 2.3, the Waf source code depends on Python 2.4. When generating Waf, a parser modifies the source code and performs the following operations:
			<itemizedlist>
				<listitem>Move the decorators into simple function calls</listitem>
				<listitem>Add the imports for the module sets</listitem>
				<listitem>Eliminate redundant spaces</listitem>
				<listitem>Strip comments to reduce the size</listitem>
			</itemizedlist>
		</para>
	</section>

	<section id="coding">
		<title>Programming constraints</title>
		<para>
			Though Waf is written in Python, additional restrictions apply to the source code:
			<itemizedlist>
				<listitem>Identation is tab-only, and the maximum line length should be about 200 characters</listitem>
				<listitem>The development code is kept compatible with Python 2.3, to the exception of decorators in the Tools directory. In particular, the Waf binary can be generated using Python 2.3</listitem>
				<listitem>The <filename>wafadmin</filename> modules must be insulated from the <filename>Tools</filename> modules to keep the Waf core small and language independent</listitem>
				<listitem>Api compatibility is maintained in the cycle of a minor version (from 1.5.0 to 1.5.9)</listitem>
			</itemizedlist>
		</para>

		<para>
			More code always means more bugs. Whenever possible, unnecessary code must be removed, and the existing code base should be simplified.
		</para>
	</section>

</chapter>