DB API

The DB library is a high performance NoSQL database for embedded applications. It supports fast, flexible local data access and transparent synchronization of data to the cloud. It is modeled after DynamoDB and the OneTable access library.

DB is a NoSQL database where database items items are documents of arbitrary complexity. Data items are implemented as JSON documents and are organized into entity tables. Application entities are defined via an entity schema that specifies data fields and attributes.

DB uses Red/black binary search indexes and has controllable local persistency to disk and to the cloud on a per-table basis.

The embedded database is a high performance NoSQL management document database.

It offers JSON document items with flexible query API with efficient import and export of database items. The database uses fast red/black binary search indexes.

Extensions

Db	Embedded Database based on DynamoDB.

Functions

void	dbAddContext(Db db, cchar name, cchar *value)
	Add global context properties.
void	dbClose(Db *db)
	Close a database.
const DbItem *	dbCreate(Db db, cchar model, Json props, DbParams params)
	Create a new item.
cchar *	dbField(const DbItem item, cchar fieldName)
	Fetch a field value from an item as a string.
bool	dbFieldBool(const DbItem item, cchar fieldName)
	Fetch a field value from an item as a boolean.
Time	dbFieldDate(const DbItem item, cchar fieldName)
	Fetch a field value from an item as a date.
double	dbFieldDouble(const DbItem item, cchar fieldName)
	Fetch a field value from an item as a double.
int64	dbFieldNumber(const DbItem item, cchar fieldName)
	Fetch a field value from an item as a number (64 bit).
RList *	dbFind(Db db, cchar model, Json props, DbParams params)
	Find matching items in the database.
const DbItem *	dbFindOne(Db db, cchar model, Json props, DbParams params)
	Find the first matching item.
const DbItem *	dbGet(Db db, cchar model, Json props, DbParams params)
	Read a matching item from the database.
const cchar *	dbGetError(Db *db)
	Get an error message for the most recent API call.
cchar *	dbGetField(Db db, cchar model, cchar fieldName, Json props, DbParams *params)
	Get a field from a matching item from the database.
DbModel *	dbGetItemModel(Db db, DbItem item)
	Get the model object for a data item.
DbModel *	dbGetModel(Db db, cchar name)
	Get the model object for a model name.
Json *	dbJson(cchar *fmt, ...)
	Parse a string into Json properties.
Json *	dbJsonProps(cchar *props[])
	Convert a list of keyword / value pairs into a JSON object.
cchar *	dbNext(Db db, RList list)
	Get the next key when using paginated find requests.
Db *	dbOpen(cchar path, cchar schema, int flags)
	Open a database.
int	dbRemove(Db db, cchar model, Json props, DbParams params)
	Remove matching items in the database.
int	dbSave(Db db, cchar filename)
	Save the database.
const DbItem *	dbSetBool(Db db, cchar model, cchar fieldName, bool value, Json props, DbParams *params)
	Set an item field value as a boolean.
void	dbSetCallback(Db db, DbCallback proc, void arg)
	Set the database change trigger callback.
const DbItem *	dbSetDate(Db db, cchar model, cchar fieldName, Time value, Json props, DbParams *params)
	Set an item field value as a date.
const DbItem *	dbSetDouble(Db db, cchar model, cchar fieldName, double value, Json props, DbParams *params)
	Set an item field value as a double.
const DbItem *	dbSetField(Db db, cchar model, cchar fieldName, cchar value, Json props, DbParams params)
	Set an item field value.
const DbItem *	dbSetNum(Db db, cchar model, cchar fieldName, int64 value, Json props, DbParams *params)
	Set an item field value as a number.
const DbItem *	dbUpdate(Db db, cchar model, Json props, DbParams params)
	Update an item.

Typedefs

DbCallback	Database callback on changes.
DbItem	Database items stored in RB indexes.
DbModel	Model schema.
DbParams	Database parameters.

Defines

#define	DB_JSON dbJson(__VA_ARGS__)
	Macro for supplying API properties as a JSON parsable string.
#define	DB_MAX_ITEM (256 * 1024)
	Maximum database item length.
#define	DB_MAX_KEY 1024
	Maximum sort key length.
#define	DB_MAX_LOG_AGE (60 * TPS)
	Maximum age of log file.
#define	DB_MAX_LOG_SIZE (1024 * 1024)
	Maximum sort key length.
#define	DB_OPEN_RESET 0x2
	Reset (erase) database on open.
#define	DB_PARAMS & (DbParams) { __VA_ARGS__ }
	Macro for supplying API parameters.
#define	DB_PROPS dbJsonProps((cchar*[]) { __VA_ARGS__, NULL })
	Macro for supplying API properties as key/value pairs.
#define	DB_READ_ONLY 0x1
	Database flags.
#define	DB_JSON dbJson(__VA_ARGS__)
	Macro for supplying API properties as a JSON parsable string.
#define	DB_PARAMS & (DbParams) { __VA_ARGS__ }
	Macro for supplying API parameters.
#define	DB_PROPS dbJsonProps((cchar*[]) { __VA_ARGS__, NULL })
	Macro for supplying API properties as key/value pairs.

Db

Embedded Database based on DynamoDB.

Description:: The DB library is a high performance NoSQL in-memory database for embedded applications modeled on DynamoDB. Data items are implemented as JSON documents and are organized into tables. Application entities are defined via an entity schema that specifies data fields and attributes. Data items are JSON documents and are accessed via a flexible API and dot notation queries. DB uses Red/black binary search indexes and has controllable persistency locally to disk and to the cloud on a per-table basis.

API Stability:: Evolving.

Fields:

DbCallback	callback	Database change notification trigger.
void *	callbackArg	Argument for the trigger callback.
int	code	API error code.
Json *	context	Global context properties - overwrites API properties.
char *	error	API error message.
int	flags	Reserved.
FILE *	journal	Journal file descriptor.
Ticks	journalCreated	When journal file recreated.
bool	journalError	Journal I/O error.
REvent	journalEvent	Timeout for journal save.
char *	journalPath	On-disk journal filename.
ssize	journalSize	Current size of journal file.
Ticks	maxJournalAge	Maximum age of journal file before saving.
ssize	maxJournalSize	Maximum size of the journal before saving.
RHash *	models	List of schema models.
char *	path	On-disk path.
RbTree *	primary	Red/black tree primary index.
Json *	schema	OneTable schema.
bool	timestamps	Maintain created/updated timestamps (if in schema).
cchar *	type	Item schema type property.

void dbAddContext (Db *db, cchar *name, cchar *value)

Add global context properties.

Description:: A global context property can be added to the set of properties supplied to each API. These property values are added at the top level only. Dotted notation is not supported.

Parameters:

db	Database instance.
name	Name of the property.
value	Property value to set.

API Stability:: Evolving.

void dbClose (Db *db)

Close a database.

Description:: This will immediately save any unsaved changes and then close the database.

Parameters:

db	Database instance returned via dbOpen

API Stability:: Evolving.

const DbItem * DbItem * dbCreate (Db *db, cchar *model, Json *props, DbParams *params)

Create a new item.

Description:: Create a new item of the required model type.

Parameters:

db	Database instance returned via dbOpen
model	Name of the schema model entity to create. The model determines the valid set of properties for the created item. Set to NULL if no model is required.
props	JSON object containing the item properties to create. Use the macro DB_PROP(name, value, ...) to specify the properties as a list of keyword / value pairs. Use DB_JSON to provide the properties as a JSON/5 string. For example: DB_PROP("name", name, "address", "12 Wishbury lane") or DB_JSON("{role: 'user'}"). If a model is provided, the properties are validated against the model and unknown property names will not be written to the database.
params	List of API parameters. Use the macro DB_PARAMS(key = value, ...) to specify. bool upsert : 1; // Update on create if present. Create on update if missing.

Returns:: The created database item. The DbItem "name" property is the indexed key value. The value "json" property contains the item values as a cached JSON object. Caller must not free the returned item. Returns null on errors. Use dbGetError to retrieve an error message.

API Stability:: Evolving.

cchar * * dbField (const DbItem *item, cchar *fieldName)

Fetch a field value from an item as a string.

Description:: Use to examine an item returned via dbGet or other APIs that return items.

Parameters:

item	Database item returned from other APIs.
fieldName	Name of the field to examine.

Returns:: The field value as a string. Caller must not free.

API Stability:: Evolving.

bool dbFieldBool (const DbItem *item, cchar *fieldName)

Fetch a field value from an item as a boolean.

Description:: Use to examine an item returned via dbGet or other APIs that return items. Will return true if the item field value is set to "true" or "1".

Parameters:

item	Database item returned from other APIs.
fieldName	Name of the field to examine.

Returns:: The field value as a boolean.

API Stability:: Evolving.

Time dbFieldDate (const DbItem *item, cchar *fieldName)

Fetch a field value from an item as a date.

Description:: Use to examine an item returned via dbGet or other APIs that return items. This requires that the date value be stored as an ISO date string.

Parameters:

item	Database item returned from other APIs.
fieldName	Name of the field to examine.

Returns:: The field value as a date in a Time value. This is the time in milliseconds since Jan 1 1970.

API Stability:: Evolving.

double dbFieldDouble (const DbItem *item, cchar *fieldName)

Fetch a field value from an item as a double.

Description:: Use to examine an item returned via dbGet or other APIs that return items. This requires that the date value be stored as an ISO date string.

Parameters:

item	Database item returned from other APIs.
fieldName	Name of the field to examine.

Returns:: The field value as a double.

API Stability:: Evolving.

int64 dbFieldNumber (const DbItem *item, cchar *fieldName)

Fetch a field value from an item as a number (64 bit).

Description:: Use to examine an item returned via dbGet or other APIs that return items.

Parameters:

item	Database item returned from other APIs.
fieldName	Name of the field to examine.

Returns:: The field value as a 64 bit integer.

API Stability:: Evolving.

RList * * dbFind (Db *db, cchar *model, Json *props, DbParams *params)

Find matching items in the database.

Parameters:

db	Database instance returned via dbOpen
model	Name of the schema model for matching items. Set to NULL if no model is required.
props	JSON object containing item properties to match. Use the macro DB_PROP(name, value, ...) to specify the properties as a list of keyword / value pairs. Use DB_JSON to provide the properties as a JSON/5 string. For example: DB_PROP("name", name, "address", "12 Wishbury lane") or DB_JSON("{role: 'user'}"). If a model is provided, the properties are validated against the model and unknown property names will not be written to the database.
params	List of API parameters. Use the macro DB_PARAMS(key = value, ...) to specify. int limit; // Limit the number of returned or removed items. cchar index; // Name of the index to use. Defaults to "primary". Currently only supports "primary". cchar next; // Next pagination token to use as the starting point for the next page of results. DbWhere where; // Where query expression callback function.

Returns:: A list of matching items. Items can be enumerated or accessed using ITERATE_ITEMS, rGetNextItem and rGetItem. Caller must free the result using rFreeList.

API Stability:: Evolving.

const DbItem * DbItem * dbFindOne (Db *db, cchar *model, Json *props, DbParams *params)

Find the first matching item.

Parameters:

db	Database instance returned via dbOpen
model	Name of the schema model for matching items. Set to NULL if no model is required.
props	JSON object containing item properties to match. Use the macro DB_PROP(name, value, ...) to specify the properties as a list of keyword / value pairs. Use DB_JSON to provide the properties as a JSON/5 string. For example: DB_PROP("name", name, "address", "12 Wishbury lane") or DB_JSON("{role: 'user'}"). If a model is provided, the properties are validated against the model and unknown property names will not be written to the database.
params	List of API parameters. Use the macro DB_PARAMS(key = value, ...) to specify. cchar *index; // Name of the index to use. Defaults to "primary". Currently only supports "primary". DbWhere where; // Where query expression callback function.

Returns:: The first matching item. Returns null if no match found.

API Stability:: Evolving.

const DbItem * DbItem * dbGet (Db *db, cchar *model, Json *props, DbParams *params)

Read a matching item from the database.

Parameters:

db	Database instance returned via dbOpen
model	Name of the schema model for matching items.
props	JSON object containing the item properties to match. Use the macro DB_PROP(name, value, ...) to specify the properties as a list of keyword / value pairs. Use DB_JSON to provide the properties as a JSON/5 string. For example: DB_PROP("name", name, "address", "12 Wishbury lane") or DB_JSON("{role: 'user'}"). If a model is provided, the properties are validated against the model and unknown property names will not be written to the database.
params	List of API parameters. Use the macro DB_PARAMS(key = value, ...) to specify. cchar *index; // Name of the index to use. Defaults to "primary". Currently only supports "primary".

Returns:: The matching database item. The DbItem "name" property is the indexed key. The value "json" property contains the item values as a cached JSON object. If null, the "value" property contains the item's value as an unparsed JSON string. Use dbField to access the individual field values in the item. Caller must not free the returned item.

API Stability:: Evolving.

const cchar * cchar * dbGetError (Db *db)

Get an error message for the most recent API call.

Parameters:

db	Database instance returned via dbOpen

Returns:: A static error message string. Caller must not free.

API Stability:: Evolving.

cchar * * dbGetField (Db *db, cchar *model, cchar *fieldName, Json *props, DbParams *params)

Get a field from a matching item from the database.

Parameters:

db	Database instance returned via dbOpen
model	Name of the schema model for matching items.
fieldName	Name of the item field to return.
props	JSON object containing the item properties to match. Use the macro DB_PROP(name, value, ...) to specify the properties as a list of keyword / value pairs. Use DB_JSON to provide the properties as a JSON/5 string. For example: DB_PROP("name", name, "address", "12 Wishbury lane") or DB_JSON("{role: 'user'}"). If a model is provided, the properties are validated against the model and unknown property names will not be written to the database.
params	List of API parameters. Use the macro DB_PARAMS(key = value, ...) to specify. cchar *index; // Name of the index to use. Defaults to "primary". Currently only supports "primary".

Returns:: A string containing the required field in the data item. Caller must not free.

API Stability:: Evolving.

DbModel * * dbGetItemModel (Db *db, DbItem *item)

Get the model object for a data item.

Parameters:

db	Database instance.
item	Data item with a model type field.

Returns:: A model instance.

API Stability:: Prototype.

DbModel * * dbGetModel (Db *db, cchar *name)

Get the model object for a model name.

Parameters:

db	Database instance.
name	Model name.

Returns:: A model instance.

API Stability:: Prototype.

Json * * dbJson (cchar *fmt, ...)

Parse a string into Json properties.

Parameters:

fmt	Printf style format string.
...	Arguments to fmt.

Returns:: A JSON object containing the supplied property values.

API Stability:: Internal.

Json * * dbJsonProps (cchar *props)

Convert a list of keyword / value pairs into a JSON object.

Parameters:

props

NULL terminated array of keyword / value pairs.

Returns:: A JSON object containing the supplied property values.

API Stability:: Internal.

cchar * * dbNext (Db *db, RList *list)

Get the next key when using paginated find requests.

Parameters:

db	Database instance.
list	A list returned from a prior dbFind request.

Returns:: A reference into the last item returned in the list.

API Stability:: Prototype.

Db * * dbOpen (cchar *path, cchar *schema, int flags)

Open a database.

Parameters:

path	Filename for from which to load and save the database when calling dbSave. On open, an initial load is performed from the file at path.
schema	OneTable data schema describing the indexes and data models.
flags	Reserved. Set to zero.

API Stability:: Evolving.

See Also:: dbClose

int dbRemove (Db *db, cchar *model, Json *props, DbParams *params)

Remove matching items in the database.

Parameters:

db	Database instance returned via dbOpen
model	Name of the schema model for matching items.
props	JSON object containing the item properties to match. Use the macro DB_PROP(name, value, ...) to specify the properties as a list of keyword / value pairs. Use DB_JSON to provide the properties as a JSON/5 string. For example: DB_PROP("name", name, "address", "12 Wishbury lane") or DB_JSON("{role: 'user'}"). If a model is provided, the properties are validated against the model and unknown property names will not be written to the database.
params	List of API parameters. Use the macro DB_PARAMS(key = value, ...) to specify. int limit; // Limit the number of items to remove. cchar *index; // Name of the index to use. Defaults to "primary". Currently only supports "primary".

Returns:: A count of the number of items removed.

API Stability:: Evolving.

int dbSave (Db *db, cchar *filename)

Save the database.

Parameters:

db	Database instance.
filename	Optional filename to save data to. If set to NULL, the data is saved to the name given when opening the database via dbOpen

API Stability:: Evolving.

const DbItem * DbItem * dbSetBool (Db *db, cchar *model, cchar *fieldName, bool value, Json *props, DbParams *params)

Set an item field value as a boolean.

Parameters:

db	Database instance returned via dbOpen
model	Name of the schema model for the item. Set to NULL if no model is required.
fieldName	Name of the item field to set.
value	Value to assign to the item's field.
props	JSON object containing the item properties to match. Use the macro DB_PROP(name, value, ...) to specify the properties as a list of keyword / value pairs. Use DB_JSON to provide the properties as a JSON/5 string. For example: DB_PROP("name", name, "address", "12 Wishbury lane") or DB_JSON("{role: 'user'}"). If a model is provided, the properties are validated against the model.
params	List of API parameters. Use the macro DB_PARAMS(key = value, ...) to specify. cchar *index; // Name of the index to use. Defaults to "primary". Currently only supports "primary".

Returns:: The updated item. Caller must not free.

API Stability:: Evolving.

void dbSetCallback (Db *db, DbCallback proc, void *arg)

Set the database change trigger callback.

Description:: When database items are changed, the trigger callback is invoked to notify regarding the change.

Parameters:

db	Database instance.
proc	Database trigger function.
arg	Argument to pass to the trigger function.

API Stability:: Evolving.

const DbItem * DbItem * dbSetDate (Db *db, cchar *model, cchar *fieldName, Time value, Json *props, DbParams *params)

Set an item field value as a date.

Parameters:

db	Database instance returned via dbOpen
model	Name of the schema model for the item. Set to NULL if no model is required.
fieldName	Name of the item field to set.
value	Value to assign to the item's field.
props	JSON object containing the item properties to match. Use the macro DB_PROP(name, value, ...) to specify the properties as a list of keyword / value pairs. Use DB_JSON to provide the properties as a JSON/5 string. For example: DB_PROP("name", name, "address", "12 Wishbury lane") or DB_JSON("{role: 'user'}"). If a model is provided, the properties are validated against the model.
params	List of API parameters. Use the macro DB_PARAMS(key = value, ...) to specify. cchar *index; // Name of the index to use. Defaults to "primary". Currently only supports "primary".

Returns:: The updated item. Caller must not free.

API Stability:: Evolving.

const DbItem * DbItem * dbSetDouble (Db *db, cchar *model, cchar *fieldName, double value, Json *props, DbParams *params)

Set an item field value as a double.

Parameters:

db	Database instance returned via dbOpen
model	Name of the schema model for the item. Set to NULL if no model is required.
fieldName	Name of the item field to set.
value	Value to assign to the item's field.
props	JSON object containing the item properties to match. Use the macro DB_PROP(name, value, ...) to specify the properties as a list of keyword / value pairs. Use DB_JSON to provide the properties as a JSON/5 string. For example: DB_PROP("name", name, "address", "12 Wishbury lane") or DB_JSON("{role: 'user'}"). If a model is provided, the properties are validated against the model.
params	List of API parameters. Use the macro DB_PARAMS(key = value, ...) to specify. cchar *index; // Name of the index to use. Defaults to "primary". Currently only supports "primary".

Returns:: The updated item. Caller must not free.

API Stability:: Evolving.

const DbItem * DbItem * dbSetField (Db *db, cchar *model, cchar *fieldName, cchar *value, Json *props, DbParams *params)

Set an item field value.

Description:: Update a field in an existing item. The item must already exist.

Parameters:

db	Database instance returned via dbOpen
model	Name of the schema model for the item. Set to NULL if no model is required.
fieldName	Name of the item field to set.
value	Value to assign to the item's field.
props	JSON object containing the item properties to match. Use the macro DB_PROP(name, value, ...) to specify the properties as a list of keyword / value pairs. Use DB_JSON to provide the properties as a JSON/5 string. For example: DB_PROP("name", name, "address", "12 Wishbury lane") or DB_JSON("{role: 'user'}"). If a model is provided, the properties are validated against the model.
params	List of API parameters. Use the macro DB_PARAMS(key = value, ...) to specify. cchar *index; // Name of the index to use. Defaults to "primary". Currently only supports "primary".

Returns:: The updated item. Caller must not free.

API Stability:: Evolving.

const DbItem * DbItem * dbSetNum (Db *db, cchar *model, cchar *fieldName, int64 value, Json *props, DbParams *params)

Set an item field value as a number.

Parameters:

db	Database instance returned via dbOpen
model	Name of the schema model for the item. Set to NULL if no model is required.
fieldName	Name of the item field to set.
value	Value to assign to the item's field.
props	JSON object containing the item properties to match. Use the macro DB_PROP(name, value, ...) to specify the properties as a list of keyword / value pairs. Use DB_JSON to provide the properties as a JSON/5 string. For example: DB_PROP("name", name, "address", "12 Wishbury lane") or DB_JSON("{role: 'user'}"). If a model is provided, the properties are validated against the model.
params	List of API parameters. Use the macro DB_PARAMS(key = value, ...) to specify. cchar *index; // Name of the index to use. Defaults to "primary". Currently only supports "primary".

Returns:: The updated item. Caller must not free.

API Stability:: Evolving.

const DbItem * DbItem * dbUpdate (Db *db, cchar *model, Json *props, DbParams *params)

Update an item.

Parameters:

db	Database instance returned via dbOpen
model	Name of the schema model for the item. Set to NULL if no model is required.
props	JSON object containing the item properties to update. Use the macro DB_PROP(name, value, ...) to specify the properties as a list of keyword / value pairs. Use DB_JSON to provide the properties as a JSON/5 string. For example: DB_PROP("name", name, "address", "12 Wishbury lane") or DB_JSON("{role: 'user'}"). If a model is provided, the properties are validated against the model and unknown property names will not be written to the database.
params	List of API parameters. Use the macro DB_PARAMS(key = value, ...) to specify. cchar *index; // Name of the index to use. Defaults to "primary". Currently only supports "primary".

Returns:: The updated item. Caller must not free. Returns null on errors. Use dbGetError to retrieve an error message.

API Stability:: Evolving.

Typedefs

typedef void(* DbCallback) (void *arg, struct Db *db, struct DbModel *model, struct DbItem *item, struct DbParams *params, cchar *cmd).

Database callback on changes.

Parameters:

arg	User argument provided to dbSetCallback.
db	Database instance.
model	DbModel reference describing the item's schema model.
item	Database item that is changing.
params	User params provided to the API that caused the change.
cmd	The nature of the change. Set to "create", "update" or "remove".

API Stability:: Evolving.

DbItem

Database items stored in RB indexes.

API Stability:: Evolving.

Fields:

uint	allocatedName	The name is allocated and must be freed when removed.
uint	allocatedValue	The value is allocated and must be freed when removed.
Json *	json	Parsed JSON value of the item.
char *	key	Indexed name of the item. Used as the sort key.
char *	value	Text value of the item (JSON string).

DbModel

Model schema.

Description:: The model schema defines an application entity and the supported entity fields.

Fields:

RHash *	fields	Hash of model fields.
cchar *	name	Name of the model.
uint	sync	Sync model items to the cloud.
Time	syncDelay	Time to delay syncing changes to the cloud.

API Stability:: Evolving.

DbParams

Database parameters.

API Stability:: Evolving.

Fields:

cvoid *	arg	Argument to where callback.
bool	bypass	Bypass change trigger.
cchar *	index	Name of index to use. Defaults to "primary". Currently only supports "primary".
int	limit	Limit the number of returned or removed items.
bool	log	Emit trace information to the log.
cchar *	next	Next pagination token to use as the starting point for the next page of results.
bool	upsert	Update on create if present. Create on update if missing.
DbWhere	where	Where query expression callback.