<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
<html>
<head>
<title>Programming in Python with Medusa and the Async Sockets Library</title>
</head>
<body>
<h1>Programming in Python with Medusa and the Async Sockets Library</h1>
<h2>Introduction</h2>
<h3>Why Asynchronous?</h3>
<p>
There are only two ways to have a program on a single processor do
'more than one thing at a time'. Multi-threaded programming is
the simplest and most popular way to do it, but there is another
very different technique, that lets you have nearly all the
advantages of multi-threading, without actually using multiple
threads. It's really only practical if your program is <i>I/O
bound</i> (I/O is the principle bottleneck). If your program is
CPU bound, then pre-emptive scheduled threads are probably what
you really need. Network servers are rarely CPU-bound, however.
</p>
<p>
If your operating system supports the <code>select()</code>
system call in its I/O library (and nearly all do), then you can
use it to juggle multiple communication channels at once; doing
other work while your I/O is taking place in the "background".
Although this strategy can seem strange and complex (especially
at first), it is in many ways easier to understand and control
than multi-threaded programming. The library documented here
solves many of the difficult problems for you, making the task
of building sophisticated high-performance network servers and
clients a snap.
</p>
<h3>Select-based multiplexing in the real world</h3>
<p>
Several well-known Web servers (and other programs) are written using
exactly this technique:
the <a href="http://www.acme.com/software/thttpd/">thttpd</a>
and <a href="http://www.zeus.co.uk/">Zeus</a>,
and <a href="http://squid.nlanr.net/">Squid Internet Object Cache</a> servers
are excellent examples..
<a href="http://www.isc.org/inn.html">The InterNet News server (INN)</a> used
this technique for several years before the web exploded.
</p>
<p>
An interesting web server comparison chart is available at the
<a href="http://www.acme.com/software/thttpd/benchmarks.html">thttpd web site</a>
<p>
<h3>Variations on a Theme: poll() and WaitForMultipleObjects</h3>
<p>
Of similar (but better) design is the <code>poll()</code> system
call. The main advantage of <code>poll()</code> (for our
purposes) is that it does not used fixed-size file-descriptor
tables, and is thus more easily scalable than
<code>select()</code>. <code>poll()</code> is only recently becoming
widely available, so you need to check for availability on your particular
operating system.
</p>
<p>
In the Windows world, the Win32 API provides a bewildering array
of features for multiplexing. Although slightly different in
semantics, the combination of Event objects and the
<code>WaitForMultipleObjects()</code> interface gives
essentially the same power as <code>select()</code> on Unix. A
version of this library specific to Win32 has not been written
yet, mostly because Win32 also provides <code>select()</code>
(at least for sockets). If such an interface were written, it
would have the advantage of allowing us to multiplex on other
objects types, like named pipes and files.
</p>
<h3>select()</h3>
<p>
Here's what <code>select()</code> does: you pass in a set of
file descriptors, in effect asking the operating system, "let me
know when anything happens to any of these descriptors". (A
<i>descriptor</i> is simply a numeric handle used by the
operating system to keep track of a file, socket, pipe, or other
I/O object. It is usually an index into a system table of some
kind). You can also use a timeout, so that if <i>nothing</i>
happens in the allotted period, <code>select()</code> will return
control to your program.
</p>
<p>
<code>select()</code> takes three <code>fd_set</code> arguments;
one for each of the following possible states/events:
readability, writability, and exceptional conditions. The last set
is less useful than it sounds; in the context of TCP/IP it refers
to the presence of out-of-band (OOB) data. OOB is a relatively unportable
and poorly used feature that you can (and should) ignore unless you really
need it.
</p>
<p>
So that leaves only two types of events to build our programs
around; <i>read</i> events and <i>write</i> events. As it turns
out, this is actually enough to get by with, because other types
of events can be implied by the sequencing of these two. It
also keeps the low-level interface as simple as possible -
always a good thing in my book.
</p>
<h3>The polling loop</h3>
<p>
Now that you know what <code>select()</code> does, you're ready
for the final piece of the puzzle: the main polling loop. This
is nothing more than a simple while loop that continually calls
<code>select()</code> with a timeout (I usually use a 30-second
timeout). Such a program will use virtually no CPU if your
server is idle; it spends most of its time letting the operating
system do the waiting for it. This is much more efficient than a
<a href="http://wombat.doc.ic.ac.uk/foldoc/foldoc.cgi?query=busy-wait">busy-wait</a>
loop.
</p>
<p> Here is a pseudo-code example of a polling loop:
<pre>
while (any_descriptors_left):
events = select (descriptors, timeout)
for event in events:
handle_event (event)
</pre>
<p>
If you take a look at the code used by the library, it looks
very similar to this. (see the file <code>asyncore.py</code>,
the functions poll() and loop()). Now, on to the magic that must
take place to handle the events...
</p>
<h2>The Code</h2>
<h3>Blocking vs. Non-Blocking</h3>
<p>
File descriptors can be in either blocking or non-blocking mode.
A descriptor in blocking mode will stop (or 'block') your entire
program until the requested event takes place. For example, if
you ask to read 64 bytes from a descriptor attached to a socket
which is ultimately connected to a modem deep in the backwaters
of the Internet, you may wait a while for those 64 bytes.
</p>
<p>
If you put the descriptor in non-blocking mode, then one of two
things might happen: if the data is sitting in a local buffer,
it will be returned to you immediately; otherwise you will get
back a code (usually <code>EWOULDBLOCK</code>) telling you that
the read is in progress, and you should check back later to see
if it's done.
<p>
<h3>sockets vs. other kinds of descriptors</h3>
<p>
Although most of our discussion will be about TCP/IP sockets, on
Unix you can use <code>select()</code> to multiplex other kinds
of communications objects, like pipes and ttys. (Unfortunately,
select() cannot be used to do non-blocking file I/O. Please
correct me if you have information to the contrary!)
</p>
<h3>The socket_map</h3>
<p>
We use a global dictionary (<code>asyncore.socket_map</code>) to
keep track of all the active socket objects. The keys for this
dictionary are the objects themselves. Nothing is stored in the
value slot. Each time through the loop, this dictionary is scanned.
Each object is asked which <code>fd_sets</code> it wants to be in.
These sets are then passed on to <code>select()</code>.
</p>
<h3>asyncore.dispatcher</h3>
<p>
The first class we'll introduce you to is the
<code>dispatcher</code> class. This is a thin wrapper around a
low-level socket object. We have attached a few methods for
event-handling to it. Otherwise, it can be treated as a normal
non-blocking socket object.
</p>
<p>
The direct interface between the select loop and the socket object
are the <code>handle_read_event</code> and <code>handle_write_event</code>
methods. These are called whenever an object 'fires' that event.
</p>
<p>
The firing of these low-level events can tell us whether certain
higher-level events have taken place, depending on the timing
and state of the connection. For example, if we have asked for
a socket to connect to another host, we know that the connection
has been made when the socket fires a write event (at this point
you know that you may write to it with the expectation of
success).
<br>
The implied events are
<ul>
<li>handle_connect.
<br>implied by a write event.
<li>handle_close
<br>implied by a read event with no data available.
<li>handle_accept
<br>implied by a read event on a listening socket.
</ul>
</p>
<p>
Thus, the set of user-level events is a little larger than simply
<code>readable</code> and <code>writeable</code>. The full set of
events your code may handle are:
<ul>
<li>handle_read
<li>handle_write
<li>handle_expt (OOB data)
<li>handle_connect
<li>handle_close
<li>handle_accept
</ul>
<p>
A quick terminology note: In order to distinguish between
low-level socket objects and those based on the async library
classes, I call these higher-level objects <i>channels</i>.
</p>
<h3>Enough Gibberish, let's write some code</h3>
<p>
Ok, that's enough abstract talk. Let's do something useful and
concrete with this stuff. We'll write a simple HTTP client that
demonstrates how easy it is to build a powerful tool in only a few
lines of code.
</p>
<pre>
<font color="800000"># -*- Mode: Python; tab-width: 4 -*-</font>
<font color="808000">import</font> asyncore
<font color="808000">import</font> socket
<font color="808000">import</font> string
<font color="808000">class</font><font color="000080"> http_client</font> (asyncore.dispatcher):
<font color="808000">def</font><font color="000080"> __init__</font> (self, host, path):
asyncore.dispatcher.__init__ (self)
self.path = path
self.create_socket (socket.AF_INET, socket.SOCK_STREAM)
self.connect ((host, 80))
<font color="808000">def</font><font color="000080"> handle_connect</font> (self):
self.send (<font color="008000">'GET %s HTTP/1.0\r\n\r\n'</font> % self.path)
<font color="808000">def</font><font color="000080"> handle_read</font> (self):
data = self.recv (8192)
<font color="808000">print</font> data
<font color="808000">def</font><font color="000080"> handle_write</font> (self):
<font color="808000">pass</font>
<font color="808000">if</font> __name__ == <font color="008000">'__main__'</font>:
<font color="808000">import</font> sys
<font color="808000">import</font> urlparse
<font color="808000">for</font> url <font color="808000">in</font> sys.argv[1:]:
parts = urlparse.urlparse (url)
<font color="808000">if</font> parts[0] != <font color="008000">'http'</font>:
<font color="808000">raise</font> ValueError, <font color="008000">"HTTP URL's only, please"</font>
<font color="808000">else</font>:
host = parts[1]
path = parts[2]
http_client (host, path)
asyncore.loop()
</pre>
<!-- Thanks to Just van Rossum for PyFontify.py! -->
</pre>
<p>
HTTP is (in theory, at least) a very simple protocol. You connect to the
web server, send the string <code>"GET /some/path HTTP/1.0"</code>, and the
server will send a short header, followed by the file you asked for. It will
then close the connection.
<p>
We have defined a single new class, <code>http_client</code>, derived
from the abstract class <code>asyncore.dispatcher</code>. There are three
event handlers defined.<ul>
<li><code>handle_connect</code>
<br>Once we have made the connection, we send the request string.
<li><code>handle_read</code>
<br>As the server sends data back to us, we simply print it out.
<li><code>handle_write</code>
<br>Ignore this for the moment, I'm brushing over a technical detail
we'll clean up in a moment.
</ul>
<p>Go ahead and run this demo - giving a single URL as an argument, like this:
<p><font color="006000"><code>$ python asynhttp.py http://www.nightmare.com/</code></font>
<p>You should see something like this:
<p>
<pre>
[rushing@gnome demo]$ python asynhttp.py http://www.nightmare.com/
log: adding channel <http_client at 80ef3e8>
HTTP/1.0 200 OK
Server: Medusa/3.19
Content-Type: text/html
Content-Length: 1649
Last-Modified: Sun, 26 Jul 1998 23:57:51 GMT
Date: Sat, 16 Jan 1999 13:04:30 GMT
[... body of the file ...]
log: unhandled close event
log: closing channel 4:<http_client connected at 80ef3e8>
</pre>
<p>
The 'log' messages are there to help, they are useful when
debugging but you will want to disable them later. The first log message
tells you that a new <code>http_client</code> object has been added to the
socket map. At the end, you'll notice there's a warning that you haven't
bothered to handle the <code>close</code> event. No big deal, for now.
<p>
Now at this point we haven't seen anything <i>revolutionary</i>, but that's
because we've only looked at one URL. Go ahead and add a few other URL's
to the argument list; as many as you like - and make sure they're on different
hosts...
<p>
<i>Now</i> you begin to see why <code>select()</code> is so powerful. Depending
on your operating system (and its configuration), <code>select()</code> can be
fed hundreds, or even thousands of descriptors like this. (I've recently tested
<code>select()</code> on a FreeBSD box with over 10,000 descriptors).
<p>
A really good way to understand <code>select()</code> is to put a print statement
into the asyncore.poll() function:
<pre>
[...]
(r,w,e) = select.select (r,w,e, timeout)
print '---'
print 'read', r
print 'write', w
[...]
</pre>
<p>
Each time through the loop you will see which channels have fired
which events. If you haven't skipped ahead, you'll also notice a pointless
barrage of events, with all your http_client objects in the 'writable' set.
This is because we were a bit lazy earlier; sweeping some ugliness under
the rug. Let's fix that now.
<h3>Buffered Output</h3>
<p>
In our <code>handle_connect</code>, we cheated a bit by calling
<code>send</code> without examining its return code. In truth,
since we are using a non-blocking socket, it's (theoretically)
possible that our data didn't get sent. To do this <i>correctly</i>,
we actually need to set up a buffer of outgoing data, and then send
as much of the buffer as we can whenever we see a <code>write</code>
event:
<pre>
<font color="808000">class</font><font color="000080"> http_client</font> (asyncore.dispatcher):
<font color="808000">def</font><font color="000080"> __init__</font> (self, host, path):
asyncore.dispatcher.__init__ (self)
self.path = path
self.create_socket (socket.AF_INET, socket.SOCK_STREAM)
self.connect ((host, 80))
self.buffer = <font color="008000">'GET %s HTTP/1.0\r\n\r\n'</font> % self.path
<font color="808000">def</font><font color="000080"> handle_connect</font> (self):
<font color="808000">pass</font>
<font color="808000">def</font><font color="000080"> handle_read</font> (self):
data = self.recv (8192)
<font color="808000">print</font> data
<font color="808000">def</font><font color="000080"> writable</font> (self):
<font color="808000">return</font> (len(self.buffer) > 0)
<font color="808000">def</font><font color="000080"> handle_write</font> (self):
sent = self.send (self.buffer)
self.buffer = self.buffer[sent:]
</pre>
<p>
The <code>handle_connect</code> method no longer assumes it can
send its request string successfully. We move its work over to
<code>handle_write</code>; which trims <code>self.buffer</code>
as pieces of it are sent succesfully.
<p>
We also introduce the <code>writable</code> method. Each time
through the loop, the set of sockets is scanned, the
<code>readable</code> and <code>writable</code> methods of each
object are called to see if are interested in those events. The
default methods simply return 1, indicating that by default all
channels will be in both sets. In this case, however, we are only
interested in writing as long as we have something to write. So
we override this method, making its behavior dependent on the length
of <code>self.buffer</code>.
<p>
If you try the client now (with the print statements in
<code>asyncore.poll()</code>), you'll see that
<code>select</code> is firing more efficiently.
<h3>asynchat.py</h3>
<p>
The dispatcher class is useful, but somewhat limited in
capability. As you might guess, managing input and output
buffers manually can get complex, especially if you're working
with a protocol more complicated than HTTP.
<p>
The <code>async_chat</code> class does a lot of the heavy
lifting for you. It automatically handles the buffering of both
input and output, and provides a "line terminator" facility that
partitions an input stream into logical lines for you. It is
also carefully designed to support <i>pipelining</i> - a nice
feature that we'll explain later.
<p>
There are four new methods to introduce:
<ul>
<li><code>set_terminator (self, <eol-string>)</code>
<br>Set the string used to identify <i>end-of-line</i>. For most
Internet protocols, this is the string <code>\r\n</code>, that is;
a carriage return followed by a line feed. To turn off input scanning,
use <code>None</code>
<li><code>collect_incoming_data (self, data)</code>
<br>Called whenever data is available from
a socket. Usually, your implementation will accumulate this
data into a buffer of some kind.
<li><code>found_terminator (self)</code>
<br>Called whenever an end-of-line marker has been seen. Typically
your code will process and clear the input buffer.
<li><code>push (data)</code>
<br>This is a buffered version of <code>send</code>. It will place
the data in an outgoing buffer.
</ul>
<p>
These methods build on the underlying capabilities of
<code>dispatcher</code> by providing implementations of
<code>handle_read</code> <code>handle_write</code>, etc...
<code>handle_read</code> collects data into an input buffer, which
is continually scanned for the terminator string. Data in between
terminators is feed to your <code>collect_incoming_data</code> method.
<p>
The implementation of <code>handle_write</code> and <code>writable</code>
examine an outgoing-data queue, and automatically send data whenever
possible.
<h3>A Proxy Server</h3>
In order to demonstrate the <code>async_chat</code> class, we will
put together a simple proxy server. A proxy server combines a server
and a client together, in effect sitting between the real server and
client. You can use this to monitor or debug protocol traffic.
<pre>
<font color="800000"># -*- Mode: Python; tab-width: 4 -*-</font>
<font color="808000">import</font> asynchat
<font color="808000">import</font> asyncore
<font color="808000">import</font> socket
<font color="808000">import</font> string
<font color="808000">class</font><font color="000080"> proxy_server</font> (asyncore.dispatcher):
<font color="808000">def</font><font color="000080"> __init__</font> (self, host, port):
asyncore.dispatcher.__init__ (self)
self.create_socket (socket.AF_INET, socket.SOCK_STREAM)
self.set_reuse_addr()
self.there = (host, port)
here = (<font color="008000">''</font>, port + 8000)
self.bind (here)
self.listen (5)
<font color="808000">def</font><font color="000080"> handle_accept</font> (self):
proxy_receiver (self, self.accept())
<font color="808000">class</font><font color="000080"> proxy_sender</font> (asynchat.async_chat):
<font color="808000">def</font><font color="000080"> __init__</font> (self, receiver, address):
asynchat.async_chat.__init__ (self)
self.receiver = receiver
self.set_terminator (None)
self.create_socket (socket.AF_INET, socket.SOCK_STREAM)
self.buffer = <font color="008000">''</font>
self.set_terminator (<font color="008000">'\n'</font>)
self.connect (address)
<font color="808000">def</font><font color="000080"> handle_connect</font> (self):
<font color="808000">print</font> <font color="008000">'Connected'</font>
<font color="808000">def</font><font color="000080"> collect_incoming_data</font> (self, data):
self.buffer = self.buffer + data
<font color="808000">def</font><font color="000080"> found_terminator</font> (self):
data = self.buffer
self.buffer = <font color="008000">''</font>
<font color="808000">print</font> <font color="008000">'==> (%d) %s'</font> % (self.id, repr(data))
self.receiver.push (data + <font color="008000">'\n'</font>)
<font color="808000">def</font><font color="000080"> handle_close</font> (self):
self.receiver.close()
self.close()
<font color="808000">class</font><font color="000080"> proxy_receiver</font> (asynchat.async_chat):
channel_counter = 0
<font color="808000">def</font><font color="000080"> __init__</font> (self, server, (conn, addr)):
asynchat.async_chat.__init__ (self, conn)
self.set_terminator (<font color="008000">'\n'</font>)
self.server = server
self.id = self.channel_counter
self.channel_counter = self.channel_counter + 1
self.sender = proxy_sender (self, server.there)
self.sender.id = self.id
self.buffer = <font color="008000">''</font>
<font color="808000">def</font><font color="000080"> collect_incoming_data</font> (self, data):
self.buffer = self.buffer + data
<font color="808000">def</font><font color="000080"> found_terminator</font> (self):
data = self.buffer
self.buffer = <font color="008000">''</font>
<font color="808000">print</font> <font color="008000">'<== (%d) %s'</font> % (self.id, repr(data))
self.sender.push (data + <font color="008000">'\n'</font>)
<font color="808000">def</font><font color="000080"> handle_close</font> (self):
<font color="808000">print</font> <font color="008000">'Closing'</font>
self.sender.close()
self.close()
<font color="808000">if</font> __name__ == <font color="008000">'__main__'</font>:
<font color="808000">import</font> sys
<font color="808000">import</font> string
<font color="808000">if</font> len(sys.argv) < 3:
<font color="808000">print</font> <font color="008000">'Usage: %s <server-host> <server-port>'</font> % sys.argv[0]
<font color="808000">else</font>:
ps = proxy_server (sys.argv[1], string.atoi (sys.argv[2]))
asyncore.loop()
</pre>
<p>
To try out the proxy, find a server (any SMTP, NNTP, or HTTP server should do fine),
and give its hostname and port as arguments:
<pre>
python proxy.py localhost 25
</pre>
<p>
The proxy server will start up its server on port <code>n +
8000</code>, in this case port 8025. Now, use a telnet program
to connect to that port on your server host. Issue a few
commands. See how the whole session is being echoed by your
proxy server. Try opening up several simultaneous connections
through your proxy. You might also try pointing a real client
(a news reader [port 119] or web browser [port 80]) at your proxy.
<h3>Pipelining</h3>
<p>
Pipelining refers to a protocol capability. Normally, a conversation
with a server has a back-and-forth quality to it. The client sends a
command, and waits for the response. If a client needs to send many commands
over a high-latency connection, waiting for each response can take a long
time.
<p>
For example, when sending a mail message to many recipients with
SMTP, the client will send a series of <code>RCPT</code>
commands, one for each recipient. For each of these commands,
the server will send back a reply indicating whether the mailbox
specified is valid. If you want to send a message to several
hundred recipients, this can be rather tedious if the round-trip
time for each command is long. You'd like to be able to send a
bunch of <code>RCPT</code> commands in one batch, and then count
off the responses to them as they come.
<p>
I have a favorite visual when explaining the advantages of
pipelining. Imagine each request to the server is a boxcar on a
train. The client is in Los Angeles, and the server is in New
York. Pipelining lets you hook all your cars in one long chain;
send them to New York, where they are filled and sent back to you.
Without pipelining you have to send one car at a time.
<p>
Not all protocols allow pipelining. Not all servers support it;
Sendmail, for example, does not support pipelining because it tends
to fork unpredictably, leaving buffered data in a questionable state.
A recent extension to the SMTP protocol allows a server to specify
whether it supports pipelining. HTTP/1.1 explicitly requires that
a server support pipelining.
<p>
Servers built on top of <code>async_chat</code> automatically
support pipelining. It is even possible to change the
terminator repeatedly when processing data already in the
input buffer. See the <code>handle_read</code> method if you're
interested in the gory details.
<h3>Producers</h3>
<p>
<code>async_chat</code> supports a sophisticated output
buffering model, using a queue of data-producing objects. For
most purposes, you will use the <code>push()</code> method to
send string data - but for more sophisticated usage you can push
a <code>producer</code>
<p>
A <code>producer</code> is a very simple object, requiring only
a single method in its implementation, <code>more()</code>. See
the code for <code>simple_producer</code> in
<code>asynchat.py</code> for an example. Many more examples are
available in the Medusa distribution, in the file
<code>producers.py</code>
<hr>
<address><a href="mailto:rushing@nightmare.com">Samual M. Rushing</a></address>
<!-- Created: Mon Jan 11 03:53:15 PST 1999 -->
<!-- hhmts start -->
Last modified: Fri Apr 30 21:42:52 PDT 1999
<!-- hhmts end -->
</body>
</html>
distrib
>
Mandriva
>
2010.0
>
i586
>
media
>
contrib-release
>
by-pkgid
>
02a377929ab7e6fcfb9133a7cae77813
>
files
>
104
