
``asynchat`` --- Asynchronous socket command/response handler
*************************************************************

This module builds on the ``asyncore`` infrastructure, simplifying
asynchronous clients and servers and making it easier to handle
protocols whose elements are terminated by arbitrary strings, or are
of variable length. ``asynchat`` defines the abstract class
``async_chat`` that you subclass, providing implementations of the
``collect_incoming_data()`` and ``found_terminator()`` methods. It
uses the same asynchronous loop as ``asyncore``, and the two types of
channel, ``asyncore.dispatcher`` and ``asynchat.async_chat``, can
freely be mixed in the channel map. Typically an
``asyncore.dispatcher`` server channel generates new
``asynchat.async_chat`` channel objects as it receives incoming
connection requests.

class class asynchat.async_chat

   This class is an abstract subclass of ``asyncore.dispatcher``. To
   make practical use of the code you must subclass ``async_chat``,
   providing meaningful ``collect_incoming_data()`` and
   ``found_terminator()`` methods. The ``asyncore.dispatcher`` methods
   can be used, although not all make sense in a message/response
   context.

   Like ``asyncore.dispatcher``, ``async_chat`` defines a set of
   events that are generated by an analysis of socket conditions after
   a ``select()`` call. Once the polling loop has been started the
   ``async_chat`` object's methods are called by the event-processing
   framework with no action on the part of the programmer.

   Two class attributes can be modified, to improve performance, or
   possibly even to conserve memory.

   ac_in_buffer_size

      The asynchronous input buffer size (default ``4096``).

   ac_out_buffer_size

      The asynchronous output buffer size (default ``4096``).

   Unlike ``asyncore.dispatcher``, ``async_chat`` allows you to define
   a first-in-first-out queue (fifo) of *producers*. A producer need
   have only one method, ``more()``, which should return data to be
   transmitted on the channel. The producer indicates exhaustion
   (*i.e.* that it contains no more data) by having its ``more()``
   method return the empty string. At this point the ``async_chat``
   object removes the producer from the fifo and starts using the next
   producer, if any. When the producer fifo is empty the
   ``handle_write()`` method does nothing. You use the channel
   object's ``set_terminator()`` method to describe how to recognize
   the end of, or an important breakpoint in, an incoming transmission
   from the remote endpoint.

   To build a functioning ``async_chat`` subclass your  input methods
   ``collect_incoming_data()`` and ``found_terminator()`` must handle
   the data that the channel receives asynchronously. The methods are
   described below.

async_chat.close_when_done()

   Pushes a ``None`` on to the producer fifo. When this producer is
   popped off the fifo it causes the channel to be closed.

async_chat.collect_incoming_data(data)

   Called with *data* holding an arbitrary amount of received data.
   The default method, which must be overridden, raises a
   ``NotImplementedError`` exception.

async_chat._collect_incoming_data(data)

   Sample implementation of a data collection rutine to be used in
   conjunction with ``_get_data()`` in a user-specified
   ``found_terminator()``.

async_chat.discard_buffers()

   In emergencies this method will discard any data held in the input
   and/or output buffers and the producer fifo.

async_chat.found_terminator()

   Called when the incoming data stream  matches the termination
   condition set by ``set_terminator()``. The default method, which
   must be overridden, raises a ``NotImplementedError`` exception. The
   buffered input data should be available via an instance attribute.

async_chat._get_data()

   Will return and clear the data received with the sample
   ``_collect_incoming_data()`` implementation.

async_chat.get_terminator()

   Returns the current terminator for the channel.

async_chat.handle_close()

   Called when the channel is closed. The default method silently
   closes the channel's socket.

async_chat.handle_read()

   Called when a read event fires on the channel's socket in the
   asynchronous loop.  The default method checks for the termination
   condition established by ``set_terminator()``, which can be either
   the appearance of a particular string in the input stream or the
   receipt of a particular number of characters.  When the terminator
   is found, ``handle_read()`` calls the ``found_terminator()`` method
   after calling ``collect_incoming_data()`` with any data preceding
   the terminating condition.

async_chat.handle_write()

   Called when the application may write data to the channel.   The
   default method calls the ``initiate_send()`` method, which in turn
   will call ``refill_buffer()`` to collect data from the producer
   fifo associated with the channel.

async_chat.push(data)

   Creates a ``simple_producer`` object (*see below*) containing the
   data and pushes it on to the channel's ``producer_fifo`` to ensure
   its transmission.  This is all you need to do to have the channel
   write the data out to the network, although it is possible to use
   your own producers in more complex schemes to implement encryption
   and chunking, for example.

async_chat.push_with_producer(producer)

   Takes a producer object and adds it to the producer fifo associated
   with the channel.  When all currently-pushed producers have been
   exhausted the channel will consume this producer's data by calling
   its ``more()`` method and send the data to the remote endpoint.

async_chat.readable()

   Should return ``True`` for the channel to be included in the set of
   channels tested by the ``select()`` loop for readability.

async_chat.refill_buffer()

   Refills the output buffer by calling the ``more()`` method of the
   producer at the head of the fifo.  If it is exhausted then the
   producer is popped off the fifo and the next producer is activated.
   If the current producer is, or becomes, ``None`` then the channel
   is closed.

async_chat.set_terminator(term)

   Sets the terminating condition to be recognized on the channel.
   ``term`` may be any of three types of value, corresponding to three
   different ways to handle incoming protocol data.

   +-------------+-----------------------------------------------+
   | term        | Description                                   |
   +=============+===============================================+
   | *string*    | Will call ``found_terminator()`` when the     |
   |             | string is found in the input stream           |
   +-------------+-----------------------------------------------+
   | *integer*   | Will call ``found_terminator()`` when the     |
   |             | indicated number of characters have been      |
   |             | received                                      |
   +-------------+-----------------------------------------------+
   | ``None``    | The channel continues to collect data forever |
   +-------------+-----------------------------------------------+

   Note that any data following the terminator will be available for
   reading by the channel after ``found_terminator()`` is called.

async_chat.writable()

   Should return ``True`` as long as items remain on the producer
   fifo, or the channel is connected and the channel's output buffer
   is non-empty.


asynchat - Auxiliary Classes and Functions
==========================================

class class asynchat.simple_producer(data, buffer_size=512)

   A ``simple_producer`` takes a chunk of data and an optional buffer
   size.  Repeated calls to its ``more()`` method yield successive
   chunks of the data no larger than *buffer_size*.

   more()

      Produces the next chunk of information from the producer, or
      returns the empty string.

class class asynchat.fifo(list=None)

   Each channel maintains a ``fifo`` holding data which has been
   pushed by the application but not yet popped for writing to the
   channel.  A ``fifo`` is a list used to hold data and/or producers
   until they are required.  If the *list* argument is provided then
   it should contain producers or data items to be written to the
   channel.

   is_empty()

      Returns ``True`` if and only if the fifo is empty.

   first()

      Returns the least-recently ``push()``ed item from the fifo.

   push(data)

      Adds the given data (which may be a string or a producer object)
      to the producer fifo.

   pop()

      If the fifo is not empty, returns ``True, first()``, deleting
      the popped item.  Returns ``False, None`` for an empty fifo.

The ``asynchat`` module also defines one utility function, which may
be of use in network and textual analysis operations.

asynchat.find_prefix_at_end(haystack, needle)

   Returns ``True`` if string *haystack* ends with any non-empty
   prefix of string *needle*.


asynchat Example
================

The following partial example shows how HTTP requests can be read with
``async_chat``.  A web server might create an ``http_request_handler``
object for each incoming client connection. Notice that initially the
channel terminator is set to match the blank line at the end of the
HTTP headers, and a flag indicates that the headers are being read.

Once the headers have been read, if the request is of type POST
(indicating that further data are present in the input stream) then
the ``Content-Length:`` header is used to set a numeric terminator to
read the right amount of data from the channel.

The ``handle_request()`` method is called once all relevant input has
been marshalled, after setting the channel terminator to ``None`` to
ensure that any extraneous data sent by the web client are ignored.

   class http_request_handler(asynchat.async_chat):

       def __init__(self, sock, addr, sessions, log):
           asynchat.async_chat.__init__(self, sock=sock)
           self.addr = addr
           self.sessions = sessions
           self.ibuffer = []
           self.obuffer = b""
           self.set_terminator(b"\r\n\r\n")
           self.reading_headers = True
           self.handling = False
           self.cgi_data = None
           self.log = log

       def collect_incoming_data(self, data):
           """Buffer the data"""
           self.ibuffer.append(data)

       def found_terminator(self):
           if self.reading_headers:
               self.reading_headers = False
               self.parse_headers("".join(self.ibuffer))
               self.ibuffer = []
               if self.op.upper() == b"POST":
                   clen = self.headers.getheader("content-length")
                   self.set_terminator(int(clen))
               else:
                   self.handling = True
                   self.set_terminator(None)
                   self.handle_request()
           elif not self.handling:
               self.set_terminator(None) # browsers sometimes over-send
               self.cgi_data = parse(self.headers, b"".join(self.ibuffer))
               self.handling = True
               self.ibuffer = []
               self.handle_request()
