Skip to content

Database Model Processing

The schema.process is a map of properties that control how data is stored in the database and how the data is processed.

The process definitions define the following attributes:

  • Where the database model will be stored: device / cloud / both.
  • In what direction the database syncronization will flow: to-device / to-cloud / both.
  • What metrics should be created from the data stream.

For each schema model, you may define an entry in the process collection. For example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
const DeviceSchema = {
    process: {
        Fault:   { 
            sync: 'up' 
            metrics: [{
                namespace: 'Embedthis/Device',
                fields: ['temperature'],
                dimensions: [{Device: 'deviceId'}]
            }]
        }
    },
    ...
}

Each map entry may contain the following properties:

Property Type Description
enable string Control where the model is stored. Set to 'cloud' for in the cloud, 'device' for on the device and 'both' if in both locations. Defaults to 'both'.
sync string Define the direction of the data synchronization. Set to 'down' for down to the device, 'up' for up to the cloud, or 'both' for bi-directional. Defaults to null.
metrics Array Array of metric definitions. See below for details.
notify boolean Issue an AWS EventBridge notification event for matching data. Defaults to false.

Database Synchronization

Ioto Device Tables takes the pain out of synchronizing device data into the cloud. It automatically and transparently synchronizes data between your devices and the cloud.

The database synchronization is full-duplex in that data can be modified in the device or in the cloud and it will be replicated to the other side. The synchronization is controllable on a per-model basis.

The "enable" property can be set to "cloud" if items of that type should exist only in the cloud database. Set the property to "device" if the items should exist only in the device. Set to "both" if they should exist in both databases. The default is "both".

The "sync" property defines the synchronization direction. Set to "up" to indicate the device data should be synchronized from the device up to the cloud. Set to "down" to indicate the cloud data should be replicated down to the device and set to "both" to replicate in both directions.

To design for effective synchronization, it is best to have the "sync" direction be either "up" or "down" and not "both". You should only select "both" for models that are not transactionally critical, as changes from the cloud or device may overwrite changes coming from the other directions.

Synchronization is done on a per-item basis and not field by field. This means that if you update a field in an item, the entire item will be updated on the peer side.

Metrics

For data that is synchronized to the cloud, you can create custom metrics from the data stream. These metrics can then be displayed or graphed in the Device Manager.

In the process section for a model, define a metrics array of metric definitions. Each definition selects a data item value to be converted to a metric.

For example: 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
const DeviceSchema = {
    process: {
        Fault:   { 
            sync: 'up' 
            metrics: [{
                namespace: 'Embedthis/Device',
                fields: [{Temperature: 'temp'}],
                dimensions: [{Device: 'deviceId'}]
            }]
        }
    },
    ...
}

This will create a temperature metric from the item's temp attribute.

Metric definitions may contain the following properties:

Property Type Description
buffer map Metric buffering directives. Defaults to null.
dimensions array Array of metric dimensions. Defaults to null.
fields array Array of metrics to create. Array metric entries may contain field attribute names or a map of a field attribute to a metric name.
namespace string Metric namespace for the metric. Must be set to 'Embedthis/Device'.
where string Conditional expression to select items for which to create metrics.

The metric namespace scopes the metrics and must be set to 'Embedthis/Device'. Other namespaces may be supported in the future.

The fields property contains an array with one or more metric definitions. Each definition may be either:

  • An item attribute name which is used to identify the value AND to name the metric.
  • A map of an item attribute name to a metric name.

For example:

1
2
3
{
    fields: ['temperature', 'status', 'speed']
}

This would create three metrics using the item's temperature, status and speed attributes. The metrics would take the same name.

Wheras:

1
2
3
{
    fields: [{Temp: 'temperature'}, {Online: 'status'}, {Level: 'speed'}]
}

This would create the metrics: Temp, Online and Level from the temperature, status and speed item attributes.

The dimensions property contains an array of dimensions that scope the metric. The elements of the dimensions array can be either field names or objects that map field names into a dimension name. If the values are quoted, they are used as literal values. If unquoted, they are regarded as database item field names.

For example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
process: {
    Fault:   { 
        sync: 'up' 
        metrics: [{
            namespace: 'Embedthis/Device',
            fields: ['temperature'],
            dimensions: [{}, {Device: 'deviceId'}]
        }]
    }
}

Where Expressions

The where property can be used to select matching items for which to create metrics. Where expressions use a query language is based on familiar Javascript expressions with some additional operators. Item attributes are expressed as variable names and literal values are expressed as JavaScript values.

For example: the expression:

1
error == "critical" && component == "PS1"

See Database Metrics for details.

Event Notification

For dedicated device clouds, you can issue AWS EventBridge events for matching database items via the notify property. The notify property is set to the EventBridge bus name. It can be set to "default" or any custom EventBridge bus name.

The database item will be passed to the EventBridge bus as the event context data using the Embedthis/Ioto as the event Source.

For example:

1
2
3
4
5
6
7
{
    Store: {
        enable: 'both',
        sync: 'up',
        notify: 'default',
    }
}