1 Server-initiated messages on the web

HTTP was designed to implement a request-and-response style of client-server architecture: the client sends a request, the server responds, and the communication is over. There is no mechanism for the server to initiate a request: it must come from the client.

For some web applications, the server needs to send the client messages, typically to notify the client that something has changed on the server side that the client needs to know about, such as a new email or post arriving. This is a common enough use-case that multiple approaches have been designed to achieve it.

The simplest approach to emulating server-initiated messages is called polling. The client polls the server by sending an anything new? request to it repeatedly; with the server responds to those polling requests with any updates since the last request. Polling is easy to implement, requiring only the same request-and-response architecture used for static pages. Polling is also problematic: it involves wasted network traffic and processing bandwidth, requires the server to track which updates each client has seen, and creates delays between updates occurring on the server and being visible to the client.

The most common non-polling approach to server-initiated messages for the web is WebSockets. HTTP is sent over TCP, which allows long-lasting bidirectional connections; WebSockets is a way of transitioning from HTTP’s request-and-response use of TCP to a more general open socket. A WebSockets connection is created by the client sending an HTTP request with a special header that means I’m planning to keep this TCP socket open but use it to send WebSocket messages in the future instead of HTTP messages. If the server responds with an acceptance of that change, then the TCP socket is used as an open connection to send arbitrary data between the client an server until either decides to close the connection.

2 Building a WebSocket sever application

Because both client and server need to handle messages initiated by the other, WebSocket code involves both code that runs on the server and code that runs on the client.

@routes.get('/ws')
async def websocket_handler(request):

    ws = web.WebSocketResponse()
    await ws.prepare(request)

    global nextid # initialized to 0 outside this function
    global allws # initialized as an empty dict outside this function
    myid = nextid
    nextid += 1
    allws[myid] = ws

    async for msg in ws:
        if msg.type == WSMsgType.TEXT:
            for other in allws:
                await allws[other].send_str(f'{myid}: {msg.data}')
        elif msg.type == WSMsgType.ERROR:
            print(f'ws {myid} received exception {ws.exception()}')

    del allws[myid]
    return ws

async def shutdown_ws(app):
    for other in tuple(allws):
        await allws[other].close()

if __name__ == '__main__':
    app = web.Application()
    app.add_routes(routes)
    app.on_shutdown.append(shutdown_ws)
    ...

function wsConnect() {
    window.sock = new WebSocket('ws://'+location.host+'/ws');
    sock.onclose = () => { alert("Server disconnected the WebSocket"); delete window.sock; }
    sock.onmessage = msg => {
        let li = document.createElement('li');
        li.append(msg.data);
        document.querySelector('ul').append(li);
        document.querySelector('form input').scrollIntoView()
    }
}
window.addEventListener('load', wsConnect);

function userInput(form) {
    if (!window.sock) return false;
    window.sock.send(form.querySelector('input').value)
    form.querySelector('input').value = ''
}

<ul></ul>
<form action="javascript:;" onsubmit="return userInput(this)"><input type="text"></form>

3 Debugging

When coding with WebSockets, it is common for a variety of response handlers to all be running, waiting for a new message, at the same time. In Python’s release mode (the default) if one of them crashes the app does not: there’s no error message or other information visible. Python also has a development mode that makes errors visible: run python3 -X dev yourfilename.py instead of python3 yourfilename.py to run in development mode.

4 Scaling up

A common action in a WebSocket server is to publish a message to all subscribing clients: that is, send the same message to many websockets. In our example chat application above, every message is published to all clients.

Sending messages involves several steps. Because WebSockets use TCP, a recipient must send an ACK back to sender to acknowledge receipt; if the ACK does not arrive, the sender must send the message again. In the best case, this means that a command like await ws.send_str("text") does the following

1. Sever sends a SEND message with the message body.
2. Time passes while the message works its way through the Internet.
3. Client receives the SEND message and sends an ACK message back.
4. Time passes while the message works its way through the Internet.
5. Sever receives the ACK and moves on with the next line of code.

In the worst case, the ACK doesn’t arrive so after a delay the server re-sends the SEND, trying several times before giving up on the client as disconnected.

The large portion of the server’s time that is spend in waiting is a significant source of inefficiency. This leads to several common ways of publishing messages:

• Send one message at a time, waiting for each one.

  This is the slowest option, but also the easiest to code and reason about.

  This approach was what we already saw in the code above:

  for other in allws:
      await allws[other].send_str(f'{myid}: {msg.data}')

• Send all of the messages, not waiting for one to ACK before SENDing the next; then wait for all of the ACKs in whatever order they arrive.

  This approach keeps the overall logic of one line of code happening before the next without requiring each of the waits to occur in series.

  We can code this using a scatter-gather approach: we create a task (the async version of a thread) for each message, scattering the work between them, and then gather them all back together later. The asyncio.gather library function does both the scattering (creating tasks) and gathering (waiting for them to complete).

  coroutines = [allws[other].send_str(f'{myid}: {msg.data}') for other in allws]
  await asyncio.gather(*coroutines)

• Send all of the messages, creating background tasks to handle the ACKs, and then move on without waiting for anything.

  This approach is definitely the most performant, and is what we want to do if we expect large numbers of messages and large numbers of clients. However, it means that we don’t know that the messages arrived before we run the next command, which could potentially make some application logic more complicated.

  We can code this by using the explicit task creation library function asyncio.create_task.

  for other in allws:
      asyncio.create_task(allws[other].send_str(f'{myid}: {msg.data}'))

  Other uses of create_task

  asyncio.create_task runs a multi-step coroutine in the background while doing something else in the main event loop. It’s great for improving the performance of code that will take time but we don’t need to wait for the result. It can also be used for other forms of concurrency, such as having a task with a timer to implement timeouts or create periodic events.

Which of these three options makes sense for a particular situation depends on how many users you expect to support, how much responsiveness and speed matters, and how important the synchronization of message delivery is.

5 Building a WebSocket client application

When accessing the national weather service in your MP, you saw that you could retrieve the contents of a Web API programmatically as well as in a browser. That is also true of WebSockets.

aiohttp handles client connections in a two-step process: first, a client session is created which handles caching DNS queries and other actions that can be shared across multiple HTTP requests. Then, that client session is used to create a WebSocket connection. Because WebSockets can send and receive separately, it is common to spawn a Task for sending messages and use a loop to receive messages.

import asyncio, aiohttp
url = ... # the URL of the websocket endpoint in a chat API

async def send_stuff(ws):
    while True:
        await ws.send_str("Ask me a question.")
        asyncio.sleep(10)

async def main():
    async with asyncio.ClientSession() as client:
        async with client.ws_connect(url) as ws:
            asyncio.create_task(send_stuff(ws))
            async for msg in ws:
                if msg.type == WSMsgType.TEXT and '?' in msg.data:
                    await ws.send_str('No.')

asyncio.run(main())