Skip to content

Web Server

The Ioto web server is a fast, compact web server designed for management web applications and serving device data.

Ioto can be configured to support the web server as an additional component or Ioto can be build to run only the web server and thus be a first-class replacement for any embedded web server. Embedthis has been developing embedded web servers for many years and the Ioto web server contains all that we have learned over this period.

The Ioto web server is unlike other enterprise or embedded web servers. It does not attempt to offer ALL HTTP features and functions. Rather, it aims to implement only the required core of HTTP/1.1 and thus deliver a tiny, fast, secure embedded web server that is exceptionally good at serving single page web applications and device data APIs.

The Ioto web server supports a cohesive feature set including: HTTP/1, HTTP/1.1, TLS/SSL, Action routines, user authentication, sessions, cookie management, request logging, and security sandboxing. Yet Ioto remains very small from 80K code and will run in as little as 200K RAM. It will serve over 3,000 requests/sec on a modest Raspberry PI.

web server

Goals

Our target use-cases for the Ioto web server were:

Our goals are:

  • Be as compact as possible
  • Enhance security by minimizing the code and feature footprint
  • Provide a simple, intuitive API for web applications

Features

The core web services include:

  • HTTP/1.1 protocol engine
  • TLS 1.3 support
  • Non-blocking socket communications
  • Multiple listen endpoints
  • Flexible configuration via a web.json5 file
  • Request routing with access control
  • Binding URLs to C functions for dynamic content rendering
  • Serve static files
  • Configurable redirections
  • Transfer encoding filter
  • Session and cookie management
  • Authentication and User management
  • File upload
  • JSON parser and query engine
  • Sandbox limits and timeouts

Core Components

The Ioto web server components are:

Component Description
Safe Portable Runtime Cross-platform, multi-threaded portable runtime. Includes services for memory allocation, dynamic module loading, safe string handling, lists, hashing, command execution, socket communications, threads, thread synchronization, thread-pool, events, timers, debug trace and logging.
Transport Security Layer (TLS) TLS protocol stack. This is a virtual interface that can selectively support a variety of TLS providers including: the MbedTLS and OpenSSL stacks.
HTTP/1.1 HTTP/1.1 HTTP protocol with keep-alive support.
Request Router Request routing with user authentication and access control.
Redirections Configurable redirections to TLS or new URL targets.
Dynamic Actions Render dynamic content with Actions that bind URLs to C functions.
Static File Handler The static File handler serves static content such as HTML pages, images and PDF files.
Upload Filter The upload filter supports multipart mime file upload.
Authentication and User Management Configurable user and password management. Control access to specific parts of the site.
Cookie and Session Management Automatic session creation and cookie management.
Transfer Encoding Filter The Chunk applies Transfer Chunk Encoding to outgoing data. Chunk encoding enables the HTTP connection to be reused for subsequent requests without having to establish a new TCP/IP connection. This can significantly improve network and application performance.
Logging Configurable request and response tracing including HTTP headers.

Ioto Quick Tour

The web server uses a web.json5 JSON/5 configuration file that specifies what ports and addresses to listen on, where to find the web pages, and how to route requests. This file is read when Ioto starts.

The Ioto web server is enabled via the config/ioto.json5 file. Set the services.web property to true.

1
2
3
4
5
{
    services: {
        web: true
    }
}

The web server configuration is controlled via the web.json5 file. Here is a simple example web configuration file:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
{
    web: {
        documents: "./site",
        listen: [
            "http://:80",
            "https://:443",
        ],
        tls: {
            certificate: "server.crt",
            key: "server.key",
        },
        redirect: [
            { status: 302, from: "http", to: "https" }
        ],
        routes: [
            { match: '/' },
            { match: '/index.html' },
            { match: '/public/' },
            { match: '/auth/' }
            { match: '/admin/', role: 'admin' },
            { redirect: '/auth/login' },
        ],
    }
}

The documents directory specifies where your web pages are located. The listen property nominates the IP:PORT endpoints on which you wish to listen, and the tls property specifies your TLS private key file and certificate file.

The redirect property ensures all HTTP traffic is redirected over TLS. And finally, the routes property defines a set of processing rules for incoming requests. These rules can specify required roles (capabilities) for a user to be authenticated to access the URL.

See Web Server Configuration for details.

Building Ioto with Web Server

The easiest way to build Ioto and enable the embedded web server is to select the local Console app. This will configure and build Ioto enabling the web server and also build the local Console device management VueJS app.

1
$ make APP=console clean build

If you only want to build Ioto and don't want the Console app, select the Empty app.

1
$ make APP=empty clean build

Starting Ioto

You can manually run ioto via:

1
make run

This will run Ioto with the -v verbose option. This will emit:

1
2
3
4
5
6
7
app: info: Starting Ioto 2.0.0, with "console" app 1.0.0, using "dev" profile
setup: info: Enabling services: db register serialize web
sync: info: Update device
register: info: Device already registered
web: info: Listening http://:9090
web: info: Listening https://:4443
app: info: Ioto ready

The -v means run in verbose mode where request to the server will be traced to the console. You can also run in debug mode via -D which emits more debug trace.

You can request a file from the server now with the curl utility:

1
curl http://localhost:9090/index.html

or using the supplied url command utility:

1
url 9090/index.html

Dynamic Web Content

Dynamic content is generated via the Ioto Action handler that binds specific URLs to corresponding C functions. The action handler is ideal for situations when you want to respond with device data or generate a dynamic response.

Previously, earlier web servers used the CGI and web frameworks like PHP or ESP to generate dynamic content. But these solutions are a poor fit for modern management applications that use SPA techniques. They also consume considerable CPU and memory resources and are not the best approach for embedded devices. Further, they expose large attack surfaces due to excess features that are not essential.

Ioto uses a more efficient direct binding from URLs to C functions. This direct binding of HTTP URL requests to C code is extremely efficient and since most device data is manage by C API — this provides the most direct path to accessing device data.

When coupled with a flexible streaming, non-blocking API, and a JSON parser and query engine, the result is a dramatically simpler and more efficient device management platform.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
static int testAction(Web *web)
{
    char  buf[ME_BUFSIZE];
    ssize nbytes;
    int   i;

    //  Read POST body data. This will block, but other fibers can still run.
    while ((nbytes = webRead(web, buf, sizeof(buf))) > 0) {
        printf("Got body data %.*s\n", (int) nbytes, buf);
    }

    //  Stream a response without buffering and with minimal memory footprint
    for (i = 0; i < 10000000; i++) {
        //  This will block as required, but other fibers can still run.
        webWriteFmt(web, "Hello World, now is: %s\n", rFormatLocalTime(0, 0));
    }
    webFinalize(web);
    return 0;
}

Ioto API

Ioto provides a flexible, comprehensive API so you can fully tailor dynamic request responses.

For example:

To generate a simple one line response:

1
webResponse(web, 200, "Hello World\n");

To set an output HTTP header with the response

1
webAddHeader(web, "X-Custom", "%d", 42);

To set the HTTP response status:

1
webSetStatus(web, 404);

To respond to a request with an error:

1
webError(web, 404, "Cannot serve request");

To write a database item (record) to the response:

1
2
DbItem *item = dbGet(db, "User", DB_PROPS("id", "u23877112"), 0);
webWriteItem(web, item);

To write an array (grid) of items to the response:

1
2
DbGrid *users = dbFind(db, "User", 0, 0);
webWriteItems(web, users);

To redirect to a new page:

1
webRedirect(web, 302, "/login");

To get the value of a request form field:

1
const char *user = webGetVar(web, "username", 0);

To set a variable in the authenticated session store:

1
webSetSessionVar(web, "username", username);

Ioto Background

The Ioto HTTP core is responsible for parsing the incoming HTTP request. A HTTP request is comprised of a request line followed by one or more HTTP headers that provide additional context.

The first line of the HTTP request specifies the HTTP operation method to use, the URI to access and the variant of the HTTP protocol to use. This typically looks like:

1
<b>GET</b> /index.html HTTP/1.1

This example is asking for the /index.html document via the GET method using the HTTP/1.1 protocol.

After the first line follow the headers. These are keyword, value pairs that control how the request will be processed. Typically there are 5-15 headers in most requests.

1
HeaderName: value

Some typical headers are:

Header Description
Authorization Authorization details including user name, realm, password digest and other authorization parameters to implement Basic and Digest authentication.
Connection Describe how the TCP/IP connection should be managed when the request completes.
Content-Length Length of any addition data with a POST request.
Content-Type Mime types the client prefers to accept in response to this request.
Cookie Cookie associated with the URI in the clients cookie cache.
Host Name to the target host to serve the request. This specifies the host name when using virtual hosting.
If-Modified-Since Only return the content if it has been modified since the date specified. Clients who have cached copies of a document (or graphics) use this header to allow the server to skip copying the document if it has not changed.
Keep-Alive Request the server to keep the connection alive so that subsequent requests can reuse the connection.

For example:

1
Connection: keep-alive

The HTTP core stores the parsed headers and their values for access by the request handlers. When all the headers have been processed Ioto proceeds to do request routing. This will occur before any associated POST data has been read from the client. POST data may be form data submitted by the client or it may be a file upload using the PUT method.

Request Routing

Ioto has a flexible request routing engine that processes client HTTP requests and validates access using a role-based access control mechanism. The routing engine is configured with a set of routes from the Ioto configuration file. When a request is received, it examines the routes in sequence and selects the first matching route to handle the request.

Handlers are responsible for receiving the request and creating a response. They generate response content based on the HTTP request URI, headers, any associated body data, and potential application session state. The output data flows through the output pipeline before transmission over the network to the client.

Ioto supports two request handlers:

  • Static file handler
  • Dynamic action handler

The static file handler serves document files from the configured documents directory.

The dynamic action handler responds to requests by invoking an associated C function to generate a dynamic response.

Fiber Coroutines

Ioto does not provide a web framework — nor do we believe this is the best approach for remote device management applications. Rather, we provide compellingly simple, streaming primitives to serve data for REST APIs and for single page applications.

To do this, Ioto adopts a unique approach of using fiber coroutines for parallelism and streaming. This yields an exceptionally fast web server that will serve over 3K requests per second on a modest Raspberry PI 4 and yet runs using only 80K of code.

The main benefit of fibers, is a non-blocking straight line procedural programming style.

A fiber coroutine is code that runs with its own stack and cooperatively yields to other fibers when it needs to wait. You can think of a fiber as a thread, but only one fiber runs at a time so there is no thread sync to worry about. For Go programmers, fibers are like Go routines. For JavaScript developers, fibers are similar to async/await.

You can read more in Fiber Coroutines, but for now, consider this sample Action routine:

JSON Parser and Query Engine

Ioto includes an integrated JSON parser, query engine and persistency to enable it to respond to HTTP requests with JSON payloads and remit responses in JSON.

Ioto Kickstart

Ioto has a fully working sample single page application called Kickstart. This is a VueJS application that serves a sample device application from Ioto.

To build the Kickstart app:

1
$ make APP=kickstart clean build