Developing with asyncio
***********************

Asynchronous programming is different from classic “sequential”
programming.

This page lists common mistakes and traps and explains how to avoid
them.


Debug Mode
==========

By default asyncio runs in production mode.  In order to ease the
development asyncio has a *debug mode*.

There are several ways to enable asyncio debug mode:

* Setting the "PYTHONASYNCIODEBUG" environment variable to "1".

* Using the "-X" "dev" Python command line option.

* Passing "debug=True" to "asyncio.run()".

* Calling "loop.set_debug()".

In addition to enabling the debug mode, consider also:

* setting the log level of the asyncio logger to "logging.DEBUG", for
  example the following snippet of code can be run at startup of the
  application:

     logging.basicConfig(level=logging.DEBUG)

* configuring the "warnings" module to display "ResourceWarning"
  warnings.  One way of doing that is by using the "-W" "default"
  command line option.

When the debug mode is enabled:

* asyncio checks for coroutines that were not awaited and logs them;
  this mitigates the “forgotten await” pitfall.

* Many non-threadsafe asyncio APIs (such as "loop.call_soon()" and
  "loop.call_at()" methods) raise an exception if they are called from
  a wrong thread.

* The execution time of the I/O selector is logged if it takes too
  long to perform an I/O operation.

* Callbacks taking longer than 100ms are logged.  The
  "loop.slow_callback_duration" attribute can be used to set the
  minimum execution duration in seconds that is considered “slow”.


Concurrency and Multithreading
==============================

An event loop runs in a thread (typically the main thread) and
executes all callbacks and Tasks in its thread.  While a Task is
running in the event loop, no other Tasks can run in the same thread.
When a Task executes an "await" expression, the running Task gets
suspended, and the event loop executes the next Task.

To schedule a callback from a different OS thread, the
"loop.call_soon_threadsafe()" method should be used. Example:

   loop.call_soon_threadsafe(callback, *args)

Almost all asyncio objects are not thread safe, which is typically not
a problem unless there is code that works with them from outside of a
Task or a callback.  If there’s a need for such code to call a low-
level asyncio API, the "loop.call_soon_threadsafe()" method should be
used, e.g.:

   loop.call_soon_threadsafe(fut.cancel)

To schedule a coroutine object from a different OS thread, the
"run_coroutine_threadsafe()" function should be used. It returns a
"concurrent.futures.Future" to access the result:

   async def coro_func():
        return await asyncio.sleep(1, 42)

   # Later in another OS thread:

   future = asyncio.run_coroutine_threadsafe(coro_func(), loop)
   # Wait for the result:
   result = future.result()

To handle signals and to execute subprocesses, the event loop must be
run in the main thread.

The "loop.run_in_executor()" method can be used with a
"concurrent.futures.ThreadPoolExecutor" to execute blocking code in a
different OS thread without blocking the OS thread that the event loop
runs in.


Running Blocking Code
=====================

Blocking (CPU-bound) code should not be called directly.  For example,
if a function performs a CPU-intensive calculation for 1 second, all
concurrent asyncio Tasks and IO operations would be delayed by 1
second.

An executor can be used to run a task in a different thread or even in
a different process to avoid blocking the OS thread with the event
loop.  See the "loop.run_in_executor()" method for more details.


Logging
=======

asyncio uses the "logging" module and all logging is performed via the
""asyncio"" logger.

The default log level is "logging.INFO", which can be easily adjusted:

   logging.getLogger("asyncio").setLevel(logging.WARNING)


Detect never-awaited coroutines
===============================

When a coroutine function is called, but not awaited (e.g. "coro()"
instead of "await coro()") or the coroutine is not scheduled with
"asyncio.create_task()", asyncio will emit a "RuntimeWarning":

   import asyncio

   async def test():
       print("never scheduled")

   async def main():
       test()

   asyncio.run(main())

Output:

   test.py:7: RuntimeWarning: coroutine 'test' was never awaited
     test()

Output in debug mode:

   test.py:7: RuntimeWarning: coroutine 'test' was never awaited
   Coroutine created at (most recent call last)
     File "../t.py", line 9, in <module>
       asyncio.run(main(), debug=True)

     < .. >

     File "../t.py", line 7, in main
       test()
     test()

The usual fix is to either await the coroutine or call the
"asyncio.create_task()" function:

   async def main():
       await test()


Detect never-retrieved exceptions
=================================

If a "Future.set_exception()" is called but the Future object is never
awaited on, the exception would never be propagated to the user code.
In this case, asyncio would emit a log message when the Future object
is garbage collected.

Example of an unhandled exception:

   import asyncio

   async def bug():
       raise Exception("not consumed")

   async def main():
       asyncio.create_task(bug())

   asyncio.run(main())

Output:

   Task exception was never retrieved
   future: <Task finished coro=<bug() done, defined at test.py:3>
     exception=Exception('not consumed')>

   Traceback (most recent call last):
     File "test.py", line 4, in bug
       raise Exception("not consumed")
   Exception: not consumed

Enable the debug mode to get the traceback where the task was created:

   asyncio.run(main(), debug=True)

Output in debug mode:

   Task exception was never retrieved
   future: <Task finished coro=<bug() done, defined at test.py:3>
       exception=Exception('not consumed') created at asyncio/tasks.py:321>

   source_traceback: Object created at (most recent call last):
     File "../t.py", line 9, in <module>
       asyncio.run(main(), debug=True)

   < .. >

   Traceback (most recent call last):
     File "../t.py", line 4, in bug
       raise Exception("not consumed")
   Exception: not consumed
