Mar 28, 2023

Using Ioto to Stream Responses

view-of-jupiter

Ioto provides a fast and efficient method to stream data to clients for use in dashboards or live data displays using standard HTTP without requiring WebSockets or other custom protocols.

This post is the fourth of a series on the Ioto embedded web server component. This post discusses streaming responses for Ioto requests.

The posts in the series are:

Background

Web servers use the HTTP request-response protocol, which means that a client (such as a web browser) sends a request to a server, and the server responds with the requested data. Web servers typically support keep-alive which allows a network connection to be reused for subsequent connections. This improves performance by reducing the connection overhead in establishing and securing the TCP/IP network connection. However, the browser and/or server will close the connection after a short duration if a subsequent request is not received within a short timeout. This is done to preserve network resources.

For embedded devices, there is often a need to provide frequent long-lived, real-time status updates to the browser or client. Normally this would require re-establishing a new network connection for each update. This can be onerous and slow, but there are several solution.

WebSockets have been utilized in such circumstances because they establish a two-way communication channel between a client and a server, allowing the server to send data to the client as soon as it becomes available without any request from the client. However, not all gateways support WebSockets, and for embedded devices, extra code and memory are needed, which can negatively affect performance and security.

There is another way, however. HTTP can stream responses when coupled with the newer browser Fetch API.

Streaming Responses from Ioto

There are two components to implement streaming responses in Ioto.

Streaming Action Routine in Ioto
Fetch call in the browser that receives progressive response records.

Streaming Action Routines

Ioto has a feature called Action Routines that allows for a direct connection between URLs and C functions. When a request is received, the corresponding C function is called upon to produce a dynamic response. The Action Routine has the capability to send a complete response or send records to the client gradually as needed, which enables it to stream data to the client.

An action routine is registered via the webAddAction call. In the example below, this binds the streamStatus function to the /status URL.

webAddAction(`ioto->webHost, "/status", streamStatus, NULL);

When a request is received, the streamStatus function is invoked.

static void streamStatus(Web *web)
{
    //  Start streaming status
    rStartEvent((REventProc) writeStatus, web, 0);

    //  Yield until complete
    rYieldFiber(0);

    //  Request now complete
}

The action routine schedules a writeStatus function to run and immediately yields to wait until the network connection is eventually closed. This ensures the connection to the client remains open and is not prematurely closed. The rYieldFiber call will save the current stack context and resume another fiber. In this case, it will be the scheduled event.

The writeStatus function will write response records as formatted JSON strings. In this example, we just write the current time each second.

The response records are JSON records, terminated by a new-line so individual JSON records can be separated in the browser.

static void writeStatus(Web *web)
{
    //  Write a time status as a JSON string
    if (webWriteFmt(web, "{\"time\": %lld}\n", rGetTicks()) < 0) {
        //  Connection lost, complete the request
        rResumeFiber(web->fiber, 0);
    } else {
        //  For this example, schedule another status update
        rStartEvent((REventProc) writeStatus, web, TPS);
    }
}

If the network connection is lost, or we wish to cease sending response records, we call rResumeFiber to resume the original streamStatus function which will complete the request.

Browser Fetch

In the browser, we need to take special care to asynchronously read response records as they are sent from Ioto.

<script>
    async function load() {
        //  Initiate the HTTP request
        let resp = await fetch('/status')

        //  Get a reader and decoder
        let reader = resp.body.getReader()
        var enc = new TextDecoder('utf-8')

        while (true) {
            let {value, done} = await reader.read()
            if (done) break
            //  Expect lines that are JSON objects
            let lines = enc.decode(value).trim('\n').split('\n')
            for (let line of lines) {
                let data = JSON.parse(line)
                //  Do something with the data
            }
        }
    }
    load()
</script>

The fetch API returns a promise that resolves to response object. From this we get a reader to progressively read response records.

Each response record is a JSON string that is encoded in an ArrayBuffer data type. When decoded, the JSON records are split at the new-line separator and each record is processed.

If the network connection is lost, done will be set to true and the loop will be exited.

Timeouts

If the network connection is idle for too long, the browser and/or Ioto will close the connection to conserve resources.

You can extend the Ioto network timeout by calling:

webExtendTimeout(web, milliseconds);

This will extend the connection timeout by the given number of milliseconds.

Summary

This design pattern provides fast, efficient method to stream data to the browser for use in dashboards or live data displays using only standard HTTP without requiring WebSockets or other custom protocols.

Using Ioto to Stream Responses

Background

Streaming Responses from Ioto

Streaming Action Routines

Browser Fetch

Timeouts

Summary

Want More Now?

Comments

{{comment.name}} said ...

Make a Comment