4.4. External data types
4.4.1. Overview
In some scenarios, implementing String or Opaque resources with the
anj_bytes_or_string_value_t structure and the ANJ_DATA_TYPE_BYTES /
ANJ_DATA_TYPE_STRING data types may be inefficient or even impossible. This
structure requires the entire resource value to be provided in a single buffer,
which forces you to:
store the resource’s value in program memory
know the resource’s size in advance
Note
You can find example of using anj_bytes_or_string_value_t in
basic object implementation tutorial.
In such cases you can use the ANJ_DATA_TYPE_EXTERNAL_BYTES /
ANJ_DATA_TYPE_EXTERNAL_STRING data types instead. They allow you to define
a callback that Anjay Lite calls to read successive chunks of the resource
value, so the value can be write in separate chunks directly to
internal Anjay Lite buffer.
Note
This change is transparent to the LwM2M Server, although payload encoding may differ. Specifically, for CBOR-based content formats (CBOR, SenML CBOR, LwM2M CBOR, and SenML-ETCH CBOR) the server must support Indefinite-Length Byte Strings and Indefinite-Length Text Strings.
Note
Code related to this tutorial can be found under examples/tutorial/AT-ExternalDataTypes in the Anjay Lite source directory and is based on examples/tutorial/BC-MandatoryObjects example.
4.4.2. Read a resource value from a file
One practical use case for these data types is reading a resource value from a
file or from the microcontroller’s external memory. In this example, you
implement the BinaryAppDataContainer
object with a Opaque Data resource. The value will be streamed from the
examples/tutorial/AT-ExternalDataTypes/libanj.a file, which is generated
during the build process.
4.4.2.1. Enable external data support
To use ANJ_DATA_TYPE_EXTERNAL_BYTES, enable external data types in your
CMakeLists.txt:
cmake_minimum_required(VERSION 3.16.0)
project(anjay_lite_at_external_data_types C)
set(CMAKE_C_STANDARD 99)
set(CMAKE_C_EXTENSIONS OFF)
set(ANJ_WITH_EXTRA_WARNINGS ON)
set(ANJ_WITH_EXTERNAL_DATA ON)
4.4.2.2. Set up external data callbacks
To handle ANJ_DATA_TYPE_EXTERNAL_BYTES, you need to define three callbacks:
a function to read data from an external source,
a function to open the external source before reading starts, and
a function to close the source once reading ends or an error occurs.
Read callback: anj_get_external_data_t
The read callback must follow this interface:
/**
* Callback to read a chunk of external data.
*
* Called by the library when encoding a resource whose type is
* @ref ANJ_DATA_TYPE_EXTERNAL_BYTES or @ref ANJ_DATA_TYPE_EXTERNAL_STRING.
* It may be invoked multiple times until the entire resource value
* has been streamed.
*
* The library guarantees sequential calls with monotonically increasing
* @p offset and no overlaps.
*
* @param buffer Output buffer to be filled with data.
* @param[in,out] inout_size
* - On input: size of @p buffer in bytes.
* - On output: number of bytes actually written.
* @param offset Absolute offset (in bytes) from the beginning of
* the resource value.
* @param user_args Application-defined pointer passed unchanged to
* every callback.
*
* @return
* - 0 if the end of the resource was reached (all data provided),
* - a negative value on error,
* - @ref ANJ_IO_NEED_NEXT_CALL if more data remains.
* In this case, the implementation must have filled the entire buffer
* (i.e., left @p inout_size unchanged).
*/
typedef int anj_get_external_data_t(void *buffer,
size_t *inout_size,
size_t offset,
void *user_args);
In this tutorial, the callback is:
static int get_external_data(void *buffer,
size_t *inout_size,
size_t offset,
void *user_args) {
struct external_data_user_args *args =
(struct external_data_user_args *) user_args;
size_t read_bytes = 0;
while (*inout_size != read_bytes) {
ssize_t ret_val =
pread(args->fd,
buffer,
*inout_size - read_bytes,
// We don't care about the off_t argument overflowing,
// because even if off_t were 32 bytes wide, an offset
// that large would still let us handle files bigger than
// the maximum file size that can be sent over CoAP
(off_t) (offset + read_bytes));
if (ret_val == 0) {
*inout_size = read_bytes;
log(L_INFO, "The file has been completely read");
return 0;
} else if (ret_val < 0) {
log(L_ERROR, "Error during reading from the file");
return -1;
}
read_bytes += (size_t) ret_val;
}
return ANJ_IO_NEED_NEXT_CALL;
}
Through the user_args argument we pass a pointer to the structure whose
definition is shown below:
static struct external_data_user_args {
int fd;
} file_external_data_args = { -1 };
This structure only holds a file descriptor that all callbacks share.
The while loop is required because pread might read fewer bytes than
requested in its count parameter, and, as specified for
anj_get_external_data_t, the implementation must fill the buffer with
exactly the number of bytes indicated by inout_size if we intend to return
ANJ_IO_NEED_NEXT_CALL.
Open callback: anj_open_external_data_t
The second callback initializes the external data source before reading:
/**
* Callback to initialize the external data source.
*
* Invoked once before the first call to @ref anj_get_external_data_t.
* Can be used to open files, initialize peripherals, or allocate state.
*
* @param user_args Application-defined pointer.
*
* @return
* - 0 on success,
* - a negative value if initialization failed (in which case
* @ref anj_close_external_data_t will not be called).
*/
typedef int anj_open_external_data_t(void *user_args);
In this example, the callback opens the file. The file path is defined by the
FILE_PATH macro:
static int open_external_data(void *user_args) {
struct external_data_user_args *args =
(struct external_data_user_args *) user_args;
assert(args->fd == -1);
args->fd = open(FILE_PATH, O_RDONLY | O_CLOEXEC);
if (args->fd == -1) {
log(L_ERROR, "Error during opening the file");
return -1;
}
log(L_INFO, "File opened");
return 0;
}
Close callback: anj_close_external_data_t
The third callback handles de-initializing the external data source and is called after all data have been read or when an error occurs:
/**
* Callback to clean up the external data source.
*
* Invoked after reading completes (successfully or with error),
* unless @ref anj_open_external_data_t failed.
* Can be used to close file descriptors, release memory, or reset state.
*
* @param user_args Application-defined pointer.
*/
typedef void anj_close_external_data_t(void *user_args);
In this example, the callback closes the file:
static void close_external_data(void *user_args) {
struct external_data_user_args *args =
(struct external_data_user_args *) user_args;
close(args->fd);
args->fd = -1;
log(L_INFO, "File closed");
}
4.4.2.3. Assign callbacks in anj_res_value_t
The addresses of the above callbacks, together with the
external_data_user_args instance address, are assigned to the corresponding
pointers in the anj_res_value_t structure. It is done in the handler that
is called during the Read operation:
static int res_read(anj_t *anj,
const anj_dm_obj_t *obj,
anj_iid_t iid,
anj_rid_t rid,
anj_riid_t riid,
anj_res_value_t *out_value) {
(void) anj;
(void) obj;
if (iid == 0 && rid == 0 && riid == 0) {
out_value->external_data.get_external_data = get_external_data;
out_value->external_data.open_external_data = open_external_data;
out_value->external_data.close_external_data = close_external_data;
out_value->external_data.user_args = (void *) &file_external_data_args;
return 0;
}
return ANJ_DM_ERR_METHOD_NOT_ALLOWED;
}
Note
The anj_open_external_data_t and anj_close_external_data_t callbacks
are optional. You can skip them if not needed.
4.4.2.4. Add an install function
Define and install the BinaryAppDataContainer object:
static int install_binary_app_data_container_object(anj_t *anj) {
static const anj_dm_handlers_t handlers = {
.res_read = res_read,
};
// Definition of resource instance
static const anj_riid_t insts[] = { 0 };
// Definition of resource
static const anj_dm_res_t res = {
.rid = 0,
.kind = ANJ_DM_RES_RM,
.type = ANJ_DATA_TYPE_EXTERNAL_BYTES,
.insts = insts,
.max_inst_count = 1
};
// Definition of instance
static const anj_dm_obj_inst_t obj_insts = {
.iid = 0,
.res_count = 1,
.resources = &res
};
// Definition of object
static const anj_dm_obj_t obj = {
.oid = 19,
.insts = &obj_insts,
.handlers = &handlers,
.max_inst_count = 1
};
return anj_dm_add_obj(anj, &obj);
}
Note
For more information on how to add an object in Anjay Lite, see this article.
4.4.2.5. Call the install function
Finally, call the install function from main:
if (install_device_obj(&anj, &device_obj)
|| install_security_obj(&anj, &security_obj)
|| install_server_obj(&anj, &server_obj)
|| install_binary_app_data_container_object(&anj)) {
return -1;
}
Note
Large files may take a long time to transfer.