Link Search Menu Expand Document
Libertas Things Documentation

Common Libraries and API

Hornet is designed to be hardware agnostic. The code can be ported to different hardwares while the API is kept the same.

Hornet offers a set of libraries and API the programmers can use.

Hornet Libraries

Bits Array 

#include "core/lib/bits_array.h"

A bits array is an array of values. It has two properties, size and value bits. Size is the length of the array, value bits is the size of the value in bits.

For example, if the value bits is 1, each element only holds two possible values, 0 or 1. If values bits is 2, the value ranges between 0 and 3.

Bits array is initialized with an existing buffer memory big enough to hold the array. It is up to the developer to manage the buffer memory and ensure the peoper size of the memory.

API Definitions 

extern void     bits_array_init(void *p, size_t size, uint8_t bits, uint16_t value);
extern uint16_t bits_array_get(void *p, uint8_t bits, size_t index);
extern void     bits_array_set(void *p, uint8_t bits, size_t index, uint16_t value);
extern size_t   bits_array_bytes(size_t size, uint8_t bits);

bits_array_init initializes a bits array, where p is the pointer to working memory, size is array size, bits is the value bits, value is the initial value.

bits_array_get gets the value at specified index.

bits_array_set sets a value to specified index.

bits_array_bytes calculate the required memory size in bytes for a bits array. Most likely you won’t need to use it.

Flash Database (Flash DB)

The Hornet flash database library is designed by Mr. Qingjun Wei. Hornet stack is highly dependent on flash DB.

Flash is a persistent storage, with some special traits:

  1. It can only be erased by page.
  2. Once erased, it can be written once. Once written, it can be rewritten a few times before next page erase. However, one can only rewrite bit “1” into “0”, but not vice versa.
  3. Flash memory has write wear off. Each page can only be erased/written for thousands of times.

Flash DB is special designed for flash storage.

Header File 

#include "core/lib/flash_db.h"

Developer Responsibility

Developers are responsible for declaring the folowing symbols referenced by hornet.h

  • FLASH_STORAGE_START_ADDRESS - The address of Flash DB start.
  • FLASH_DB_STORAGE_PAGES - The number of pages for Flash DB. Note, FLASH_PAGE_SIZE is the size of flash page in bytes.

Read “Flash Memory Layout” and “more developer’s responsibilities” for more details.

Data Type

Flash DB is designed to manage instances of “C” data structures. Two types of storages are supported.

  1. Single record - Only one instance of data record (or NULL if no record) can be stored
  2. List of records - A list of instances of data with index will be managed. Like a database table.

Operations

Flash DB is designed for flash with unified memory address space. Each record is directly referenced by a pointer to the memory address.

  • Single record
    • Write
    • Read/Query
    • Erase
  • List of records - a list of instances of data will be managed
    • Insert (Unindexed Write) - It will insert the record into the list. A random available index is assigned to the record. If the list is full, the operation fails.
    • Indexed Write/Rewrite - It will insert the record at the specified index. If a record already exists at the specified index, record will be rewritten.
    • Get Record Index - Given a pointer to a record, it returns the index of the record. Please note the API always returns a value even on an invalid pointer.
    • foreach iteration - Iterates every record within the list by repeatedly passing each record to the specified callback function. Note the record can be erased within the callback.
    • find first - Iterates every record within the list by repeatedly passing each record to the specified callback function until a record is “found”. The loop will be stopped if a record is “found”.
    • erase single record - Erase the record represented by the pointer.
    • erase list - Erase the entire list.

Data Type ID

Each record type is assigned a unique ID within the firmware.

  • Single record type ID must be less than …
  • List record type ID must be between … and …

API

Define Type ID

A flash DB ID is called FID. Special symbol named shall be reserved for FID definition.

  • A FID definition must be prefixed with “FID_”
  • A FID definition for a C type must be “FIX_” following the C type name.

For example, FID for C struct “security_network_key_store_t” must be defined as “FID_security_network_key_store_t”.

More examples are shown below. It defines FID for a single record (mac_pib_t) and a list record (nwk_binding_device_entry_t).

#define FID_mac_pib_t 		                1
#define FID_nwk_binding_device_entry_t      4098

Init FID

The device firmware must register all valid FIDs. The list can be declared like below.

uint8_t flash_db_init_hornet() {
    DECLARE_FLASH_DB(hornet)
        DECLARE_FLASH_RECORD_DB(mac_pib_t)
        DECLARE_FLASH_LIST_RECORD_DB(nwk_binding_device_entry_t)
    END_FLASH_DB()
    return FLASH_DB_REGISTER(hornet);
}

Note, a FID list begins with DECLARE_FLASH_DB and ends with END_FLASH_DB(). Note DECLARE_FLASH_DB requires a name. The name shall be random as long as it is unique.

List FID Alias

FID definition is directly related to type. But what if there are more than one list for a given type? For exmaple, for device binding records, there should be two lists, one “incoming” and one “outgoing”.

In this case, we can have an FID alias with any name as long as it is syntactically valid, such as,

#define FID_nwk_binding_table_outgoing      4099
#define FID_nwk_binding_table_incoming      4100

We use DECLARE_FLASH_LIST_RECORD_DB_ALIAS to register alias FID of spefic type. Code below registers two FID aliases FID_nwk_binding_table_outgoing and FID_nwk_binding_table_incoming with the same data type of nwk_binding_table_entry_t.

Note FID_ prefix is not presented in the C macro.

uint8_t flash_db_init_hornet() {
    DECLARE_FLASH_DB(hornet)
        DECLARE_FLASH_RECORD_DB(mac_pib_t)
        DECLARE_FLASH_LIST_RECORD_DB(nwk_binding_device_entry_t)
        DECLARE_FLASH_LIST_RECORD_DB_ALIAS(nwk_binding_table_outgoing, nwk_binding_table_entry_t)
        DECLARE_FLASH_LIST_RECORD_DB_ALIAS(nwk_binding_table_incoming, nwk_binding_table_entry_t)
    END_FLASH_DB()
    return FLASH_DB_REGISTER(hornet);
}

Init Flash DB

flash_db_init() shall be called to initialize the flash DB.

Please note, all FIDs shall be registered before flash_db_init() is called. flash_db_init_hornet() is the built-in API to initialize hornet related FID. You shall implement your own flash_db_init_app() to register you DB data types.

    flash_db_init_hornet();
    flash_db_init_app();
    flash_db_init();

Single Record

As long as FID is defined and registered. The C type name can be used in macros and functions to access the corresponding C record from flash DB.

Write Single Record

FLASH_DB_WRITE_SINGLE_RECORD(t,r), where t is the C type name, r is a pointer to instance. For example

const mac_pib_t *p = FLASH_DB_WRITE_SINGLE_RECORD(mac_pib_t, pib);    // pib is a C pointer

Note a new pointer points to a flash region will be returned. NULL is returned if the operation failed. It always rewrites if the record already exists.

Read/Query Single Record

FLASH_DB_GET_SINGLE_RECORD(t), where t is the C type name. Please note the returned pointer may be NULL is the record doesn’t exist.

mac_pib_nvram_t *p = FLASH_DB_GET_SINGLE_RECORD(mac_pib_nvram_t);

Note the returned pointer could be NULL if no record is stored.

Remove Single Record

FLASH_DB_REMOVE_SINGLE_RECORD(t) where t is the type, for example,

FLASH_DB_REMOVE_SINGLE_RECORD(mac_pib_nvram_t);

Code above will remove single record of type mac_pib_nvram_t from flash DB.

List Record

Insert (Unindexed Write)

FLASH_DB_ADD_LIST_RECORD(t,r,m), where t is the C type name, r is a pointer to instance, m is the maximum capacity. For example,

const nwk_binding_device_entry_t *p_stored = (const nwk_binding_device_entry_t *)FLASH_DB_ADD_LIST_RECORD(nwk_binding_device_entry_t, p_ram, 64);

FLASH_DB_ADD_LIST_RECORD_ALIAS(id,t,r,m), where id is the alias name, t is the C type name, r is a pointer to instance, m is the maximum capacity. For example,

const nwk_binding_table_entry_t * p_flash = (const nwk_binding_table_entry_t *)
    FLASH_DB_ADD_LIST_RECORD_ALIAS(
        nwk_binding_table_outgoing, 
        nwk_binding_table_entry_t,
        p_ram,
        NWK_MAX_BIND_ENTRY);

It returns a pointer to the record in flash memory, or NULL indicating error.

Get Record Index

uint16_t flash_db_list_record_index(const void *record), where the record argument points a record in flash memory.

Note this function always returns a value even if record pointer is damaged or invalid.

Indexed Write/Rewrite

FLASH_DB_SET_LIST_RECORD(t,r,i), where t is the C type name, r is a pointer to instance, i is the index. For example,

const nwk_binding_device_entry_t *p_stored = (const nwk_binding_device_entry_t *)FLASH_DB_SET_LIST_RECORD(nwk_binding_device_entry_t, p_ram, 0);

FLASH_DB_SET_LIST_RECORD_ALIAS(id,t,r,i), where id is the alias name, t is the C type name, r is a pointer to instance, i is the index. For example,

const nwk_binding_table_entry_t * p_flash = (const nwk_binding_table_entry_t *)
    FLASH_DB_SET_LIST_RECORD_ALIAS(
        nwk_binding_table_outgoing, 
        nwk_binding_table_entry_t,
        p_ram,
        index);

It returns a pointer to the record in flash memory, or NULL indicating error. If record exists at the index and it’s different, it will be rewritten.

Replace Record

Replace is equivalent to FLASH_DB_SET_LIST_RECORD with the index (see flash_db_list_record_index) of the old record.

FLASH_DB_REPLACE_LIST_RECORD(t,or,nr)
FLASH_DB_REPLACE_LIST_RECORD_ALIAS(id,t,or,nr)

where t is the C type name, or is the pointer to old record (in flash), nr is the pointer to new record. For alias version, id is the alias name.

Remove A List Record

FLASH_DB_REMOVE_LIST_RECORD(t,r) where t is the C type name, r is a pointer to instance.

Please note the t is the actual C type name. It must be supplied to determine the record size.

FOR_EACH iteration 
FLASH_DB_FOR_EACH_LIST(t,f,p)
FLASH_DB_FOR_EACH_LIST_ALIAS(id,t,f,p)

where t is the C type name, f is iteration function, p is a pointer to any data. For alias version, id is the alias name.

Iteration function f must have a signature below:

typedef void (*flash_db_for_each_func_t)(const void *r, void *p);

Iteration function f will be repeatedly called on every list record, with r as the record pointer and p as the custom data pointer passed to the FOR_EACH macro.

FIND_FIRST iteration 
FLASH_DB_FIND_LIST(t,f,p)
FLASH_DB_FIND_LIST_ALIAS(id,t,f,p)

where t is the C type name, f is iteration function, p is a pointer to any data. For alias version, id is the alias name.

Iteration function f must have a signature below:

typedef const void *(*flash_db_find_list_func_t)(const void *r, const void *p);

Iteration function f will be repeatedly called on every list record, with r as the record pointer and p as the custom data pointer passed to the macro.

The iteration will stop if the iteration function returns a non-null record pointer. The record pointer will be the result.

Flash DB Page Management and Cautions

The flash DB will round-robin the flash page usage to level the write wearing.

Under rare circumstances, the records may be moved to another location to “compact” pages. If record pointers are “cached” during startup, the relocation may render the cached pointers no longer valid.

Flash DB provides mechanism to deal with the record relocations. Immediately after the relocation happened, a callback function will be called. Developer is responsible for implementing the function.

#define FLASH_DB_GC_SINGLE     (0x01)
#define FLASH_DB_GC_LIST       (0x02)
extern void flash_db_on_gc_app(uint8_t gc_flags);

As a rule of thumb, record pointer caching must be performed on flash_db_on_gc_app function. The function shall be called during start up to initialize cached record pointers. It may be called by Flash DB library when relocation happens.

Real-time Clock (RTC) API

Real-time Clock API helps maintaining a real-time clock with date and time on embedded systems.

#include "core/lib/rtc_time.h"

Declarations are shown below.

typedef struct  {
  uint8_t seconds;
  uint8_t minutes;
  uint8_t hours;
  uint8_t weekday;   // day of week, sunday is day 1
  uint8_t monthday;
  uint8_t month;
  uint8_t year;     // offset from 1970;
  uint8_t dyear;    // day of year
} qw_tm;

extern void   rtc_parse(time_t t, qw_tm *tm);
extern time_t rtc_make(qw_tm *tm);

rtc_parse behaves like gmtime or localtime in POSIX.

rtc_make behaves like mktime.

It is up to the developer to maintain the proper current time since Unix epoch.

Button API

Buttons are essential parts of devices. As far as I know there hasn’t been a good open source button driver so far.

Although buttons are hardware related. We need to design a button API with good abstraction that meets requirement below.

  • Simple and versatile. Following events shall be generated!
    • Click
    • Hold
  • Automatically reject button jitters.
  • Battery efficient. It shall allow the device to continue sleeping while the button state is not changed (even if a button stays “on-hold”).

Button API is timeing dependent. It depends on the high resolution system tick API. However, all the implementation details are hidden. Developers don’t need to deal with timing at all.

#include "core/lib/button.h"

Declarations 

#define BUTTON_STATE_UP             0x00
#define BUTTON_STATE_DOWN           0x01
#define BUTTON_STATE_CLICK          0x02
#define BUTTON_STATE_HOLD           0x04

extern void    buttons_init();
extern uint8_t button_state_update(uint8_t index, uint32_t ticks, uint8_t down);

Usage

It is suprisingly simple. Only two function calls.

Initialization

TOTAL_BUTTONS shall be defined in a C header referenced by hornet.h.

buttons_init() shall be called only once during device initialization process.

Polling Button State

Button API works only in polling mode. button_state_update shall be called every time in each polling iteration with the hardware state of the button.

index is the index of the abstract button. ticks is the current system ticks from qw_clock_ticks(). down is “1” if the hardware button is in down state, otherwise “0”.

The return value is a bit mask. Note,

  • A button may be in BUTTON_STATE_DOWN state alone or it may be a combination of BUTTON_STATE_DOWN and BUTTON_STATE_HOLD. It can only be in BUTTON_STATE_HOLD state after it’s down for a while (500ms).
  • Because of the jitter rejection, while in BUTTON_STATE_HOLD state, the button may be temporarily without BUTTON_STATE_DOWN (for less than 50ms).

Programmers shall test different states to generate on and off signals (optionally hold) depends on the button types.

  • Momentum Button - Like door bell button
    Test BUTTON_STATE_DOWN for “on” and BUTTON_STATE_CLICK for “off”.
  • Normal Button
    Test BUTTON_STATE_CLICK for “on”, “off” (or “toggle”) and BUTTON_STATE_HOLD for “hold” (optionally).
    if BUTTON_STATE_HOLD is enabled, also test == BUTTON_STATE_UP for “stop” signal.

Third party Libraries

Libraries are open source third party libraries for some common algorithms.

Contiki Project

Contiki is an open-source, cross-platform operating system. Although we use FreeRTOS, we copied a lot of code related to ARM MCU control.

The contiki code are located in /cpu folder.

BSD Queue 

#include "bsd_queue.h"

It is the renamed queue.h is from BSD source code, which implements several different linked-list or double-linked-list data structures.

For more details please read https://www.freebsd.org/cgi/man.cgi?query=queue.

Priority Queue 

#include "heap-inl.h"

It is used in our high resolution timer implementation. It is also used by libuv.

Sort - qsort_hornet

A qsort_hornet function is defined to behave consistent with qsort_r across MCU, Linux and Windows.

See https://en.cppreference.com/w/c/algorithm/qsort for more information about qsort_r.

Please note, on MCU qsort_hornet is recursive for size > 10, so be careful with stack usage.